Methods for training saturation-compensating predictors of affective response to stimuli

ABSTRACT

Described herein are methods for training a machine learning-based predictor of affective response to stimuli. The methods involve receiving samples comprising temporal windows of token instances to which a user was exposed, and target values representing affective response annotations of the user in response to the temporal windows of token instances. This data is used for the training of the predictor along with values indicative of the number of the token instances in the temporal windows of token instances, which are used to compensate for non-linear effects resulting from saturation of the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 61/484,673, filed May 11, 2011.

The following co-pending U.S. patent applications, filed on Jun. 6,2011, may include related subject matter: Ser. Nos. 13/168,960,13/168,962, 13/168,964, 13/168,967, 13/168,968, 13/168,971, 13/168,972,and 13/168,973.

BACKGROUND

The interest in the area of affective computing has grown dramaticallyin recent years. This interdisciplinary field, which spans computerscience, cognitive science, and psychology, involves the study anddevelopment of systems that can recognize, interpret, process, and evensimulate expression of human emotions. Presently, most affectivecomputing systems are still research-grade endeavors that are typicallynot robust enough to handle the demands of real world applications. Thatbeing said, the continuing increase in computing power coupled with theminiaturization and proliferation of sensors and mobile devices, ismaking widespread adoption of affective computing systems in real world,day-to-day situations, closer than ever.

One useful capability for many affective computing applications is theability to foretell a user's response to stimuli. Having this abilitycan enable the applications to offer a better user experience. Whilecurrently there are some systems that learn to predict a user's responseto stimuli, they are mostly inadequate when it comes to real worldapplications. Existing systems are usually trained on data collected ina controlled environment. In these laboratory-like settings, a smallnumber of short experiments are conducted (typically less than an hourlong), in which users' responses are measured to a set of pre-selectedstimuli, such as pictures, video scenes, or music. One drawback of thelaboratory-collected data is that it is acquired over a short period oftime, and the user is typically exposed to one stimulus at a time.However, in reality, a user's reaction to stimuli may vary dramaticallydepending on the situation the user is in, making thelaboratory-collected data less useful. For example, a user's responsewhile driving in busy traffic might be quite different from the user'sresponse when relaxing at home, even if exposed to the very similarstimuli in both situations. Furthermore, with short experiments, auser's reaction can only be measured to a small number of stimuli, whichis often inadequate for creating an affective computing system for realworld applications that may have to model the effect of a wide range ofstimuli from multiple sources. For example, a user may be exposed tomultiple stimuli coming from digital media, such as video images andsound, while at the same time, the user may be exposed to physiologicalsensation stimuli originating from the massage chair the user is sittingin. The simultaneous exposure to multiple stimuli might have non-lineareffects on the user, so for instance, the user's response might notadd-up to the response predicted for each individual stand-alonestimulus. With the many challenges and complications involved in thereal world domain, a system designed to predict a user's response tostimuli in real world scenarios may need to take into account the addedcomplexity intrinsic to this domain in order to achieve optimal results.

BRIEF SUMMARY

Affective computing systems are becoming an imminent reality. One way inwhich affective computing is bound to have substantial influence onusers' lives is through applications that tailor the way computerizedsystems interact with users in order to achieve a desired user response.For example, such systems might modify the way a robot moves or speaksso it does not annoy the user. In another example, a system may generatedifferent media content; possibly even rendering it or modifying it onthe fly, in order to increase the user's enjoyment. For a system to ableto make these adjustments optimally, it is very helpful if it has theability to predict the user's response, in real world situations, tomultiple, possibly simultaneous, stimuli such as images, sounds, words,scents, flavors or physical sensations.

To this end, some embodiments of this invention disclose a machinelearning-based predictor to predict a user's response after beingexposed to stimuli. The machine learning-based predictor is trained ondata acquired from monitoring the user over long periods, in which theuser is in many different situations. During these times, the user maybe exposed to many, sometimes simultaneously occurring, stimulioriginating from various sources. In addition to monitoring the stimuli,the user's response to the stimuli is also monitored. This response canbe expressed in different ways, for example, an affective (emotional)response or a behavioral response. By training a machine learningpredictor on real world data, it is possible for the predictor toaccount for various phenomena such as the non-linear effects ofsimultaneous exposure to multiple stimuli.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments are herein described, by way of example only, withreference to the accompanying drawings. In the drawings:

FIG. 1 illustrates an affective response library from machine learning;

FIG. 2 illustrates a measurement library from machine learning;

FIG. 3 illustrates an affective response library with input base dynamicanalysis;

FIG. 4 illustrates a measurement library with input base dynamicanalysis;

FIG. 5 illustrates a statistics based system for generating a library ofassociation between tokens and annotations;

FIG. 6 illustrates a statistics based system for generating a library ofassociation between tokens and user measurements;

FIG. 7 illustrates training an emotional state model;

FIG. 8 illustrates an emotional state predictor;

FIG. 9 illustrates training a user measurement model;

FIG. 10 illustrates a predictor for user measurements;

FIG. 11 illustrates a token database;

FIG. 12 illustrates a machine having a memory containing data used totrain a machine learning based affective response model for a user;

FIG. 13 illustrates a method in accordance with one embodiment;

FIG. 14 a illustrates a machine learning based baseline calculator;

FIG. 14 b illustrates a situation based baseline calculator

FIG. 14 c illustrates a window similarity based baseline calculator

FIG. 15 illustrates an affective response predictor for token stream;

FIG. 16 illustrates an affective response predictor for token streambased on HMM;

FIG. 17 illustrates an EM-based situation discovery;

FIG. 18 illustrates a clustering based situation discovery; and

FIG. 19 illustrates an ML based situation predictor.

DETAILED DESCRIPTION

Some of the disclosed embodiments comprise a system that monitors a userover long periods and in different situations to generate a model of theuser's affective response. Optionally, there are two main types ofinputs to the system: (i) user measurements of various types ofmodalities, and (ii) a stream of token instances that represent theuser's cognitive state and various sensual stimuli to which the user wasexposed. The system processes the user measurements to learn a model forestimating the user's affective state. By coupling the affective stateestimations for various time points with information on the tokeninstances to which the user was exposed at those times, and applyingvarious machine learning and/or data mining techniques, the systemlearns an affective response model for the user. Using the machinelearning models the system learnt, it may generate one or more of thefollowing outputs: (i) user's model parameters, which describe theuser's expected affective response to various tokens, and/or thedynamics of the user's affective response, (ii) a predictor of theuser's affective response to a token or stream of tokens, (iii) a methodfor decomposing the user's affective response to determine thecontribution of tokens and/or token sources to the affective response,and (iv) a method for emotional state annotation based on usermeasurements and tokens.

TERMS

In one embodiment, the system processes user measurement channels and/ortokens. The user measurement channels are data obtained from monitoringa user. The tokens include one or more of the following types ofinformation: (i) data that pertains to the sensual stimuli to which theuser is exposed, (ii) data that pertains to the user's cognitivecondition, and/or (iii) data that pertains to the user's physiologicalcondition. Optionally, the system may store and/or utilize datapertaining to the user's interest level in some tokens. A more detailedand comprehensive explanation about the different types of datacollected and processed by the system is provided below.

The terms “affect” and “affective response” refer to the physiologicaland/or behavioral manifestation of an entity's emotional state. Theterms “affective response/state” and “emotional response/state” may beused herein interchangeably, but usually the affective response isderived from actual measurements or observations, while the emotionalstate is predicted from models.

The term “user measurement channels”, or the alternative form“measurement channels of the user”, refer to physiological and/orbehavioral measurements of the user, which may be either rawmeasurements and/or processed measurements (e.g., resulting fromfiltration, calibration, and/or feature extraction). Examples ofphysiological measurements include various types of signals taken of theuser's physiological state using sensors for physiological properties,such as heart-rate (HR), Blood-Volume Pulse (BVP), Galvanic SkinResponse (GSR), Skin Temperature (ST), respiration,electroencephalography (EEG), electrocardiography (ECG),electromyography (EMG), Electrodermal Activity (EDA), and others.Examples of behavioral monitoring include measurements derived from oneor more cameras, microphones, movement sensors, acoustic sensors, and/orpressure sensors. The user measurements may utilize various existing,and/or yet to be invented, sensors and measurement devices that may beattached to the body, clothing (such as gloves, shirts, helmets),implanted in the user's body, and/or remote sensors external to theuser's body. It is noted that the user measurement channels are oftenreferred to in the literature as “modalities”. In one embodiment, theuser measurement channels may be received by the system as raw data,and/or after filtration (e.g., noise cancellation), and/or analyzed data(e.g., after speech recognition or image analysis).

The term “token” refers to the abstract concept of a thing that may havean influence on the user's affective state. Optionally, tokens may becategorized according to their source with respect to the user: externalor internal tokens. In one embodiment, the tokens may include one ormore of the following:

(i) Information referring to the concept of a sensual stimulus or agroup of sensual stimuli that may be experienced or sensed by the user.These tokens usually have a specified source such as objects or systemsin the user's vicinity or that the user is interacting with in some way,such as digital or printed media, augmented reality devices, roboticsystems, food, and/or beverages. For example, a token may be an item(e.g. car), a movie genre (e.g., “comedy”), a type of image (e.g.,“image of person”); a specific character (e.g., “Taco Bell Chihuahua”);web-site (e.g., “Facebook”); Scents or fragrances (e.g., “Chanel no.5”); a flavor (e.g., “salty”), a physical sensation (e.g., “pressure onthe back”).

(ii) Properties or values derived from a stimulus or group of stimuli.For example, the rate in which scenes change in a movie; the soundenergy level; the font-size in a web-page; the level of civility inwhich a robot conducts its interaction with a user.

(iii) Information about the environmental conditions that may influencethe user's affective response. For example, a token may refer to theuser's location (e.g., home vs. outdoors), the time of day, lighting,general noise level, temperature, humidity, speed (for instance, whentraveling in a car).

(iv) Information about the user's physiological and/or cognitive state.For example, the user's estimated physical and/or mental health, theuser's estimated mood and/or disposition, the user's level of alertnessand/or intoxication.

(v) Information that if the user becomes aware of it, is expected tochange the user's affective response. For example, such information mayinvolve situations where the user's child is late to coming home, theuser is in financial debt, a death in the family, depressing news, orwhen the user's mother in law is sitting next to him/her in the room.

The term “token instance” refers to the manifestation of a token duringa defined period of time or event. The relationship between a token andits instantiation (i.e., the token instance) is somewhat similar to therelationship between a class and its object in a programming language.For example, a movie the user is watching is an instance of the token“movie” or “The Blues Brothers Movie”; an image of a soda can viewedthrough a virtual reality enabled device is a token instance of “sodacan”; the sound of the soda can opening in the augmented reality videoclip played when viewing the can may be considered a token instance of“soda can popping sound”; the scent of Chanel 5 that the user smelt in adepartment store while shopping for a present is an instance of thetoken “perfume scent”, or a more specific token may be “scent of Chanelno. 5”; the temperature in the room where the user is sitting may beconsidered an instance of the token “room temperature”; the indicationthat the user sitting alone in the room is an instance of the token“being alone”, and the indication that the user is suffering from theflu may be considered an instance of the token “sick”.

The term “token source” refers to an element that influences the user'saffective state via the user's exposure to the element's tokeninstances. The token instances may be generated by the token source(e.g., a robot providing token instances describing its operations), bythe system (e.g., the system performs semantic analysis to voiceconversation), and/or received from a third party (e.g., the systemaccesses token instance repository for multimedia the user was exposedto). The term “distinct token sources” refers to token sources that areboth distinguishable from the user's perspective and operate essentiallyindependently. For example, a massage robot, a television, and a wordprocessing software operating simultaneously are distinct token sources,while audio and video stimuli generated by a computer game areconsidered as originating from the same token source. In one embodiment,a token instance may be associated with one or more sources. Optionally,a token instance without a defined token source may be attributed to anarbitrary token source, such as the environment.

The term “exposure” in the context of a user being exposed to tokeninstances means that the user is in a position to process and/or beinfluenced by the token instances, be they of any source or type.

The term “temporal window of token instances”, also referred to as“temporal token instance window” and/or “window”, refers to a set oftoken instances and other optional values, which correspond to atemporal scope defined by the window. For example, the window maycontain the token instances that at least some portion of theirexistence occurred in the window's duration. In another example, thetemporal window of token instances is a snapshot of the token instancesthat existed in a specific time point. Optionally, the window may have afixed duration. Optionally, the window may have a variable length, forexample spanning a certain event, such as the user's viewing of acommercial, visiting a web site, interacting with a robot, or reading anarticle. Optionally, the window may include values derived from othersources such as user measurement channels.

The term “situation”, in the context of the user's exposure to tokeninstances, refers to a combination of circumstances that influences theuser's affective response. Monitoring the user over a long period, andin a diverse set of day-to-day situations, reveals variations in theaffective response that are circumstance-dependent, which may not berevealed when monitoring the user over a short period or in a narrow setof similar situations. Examples of different situations may involvefactors such as: presence of other people in the vicinity of the user(e.g., being alone may be a different situation than being withcompany), the user's mood (e.g., the user being depressed may beconsidered a different situation than the user being happy), the type ofactivity the user is doing at the time (e.g., watching a movie,participating in a meeting, driving a car, may all be differentsituations). In some examples, different situations may be characterizedin one or more of the following ways: (i) the user exhibits a noticeablydifferent affective response to some of the token instances, (ii) theuser is exposed to significantly different subsets of tokens, (iii) theuser has a noticeably different user emotional state baseline value,(iv) the user has a noticeably different user measurement channelbaseline value, and/or (v) samples derived from temporal token instancewindows are clustered, and samples falling into the same cluster areassumed to belong to the same situation, while samples that fall indifferent clusters are assumed to belong to different situations.

The term “machine learning algorithm” refers to a method that evolvesits behavior based on empirical data.

Data Acquisition

User Measurements

In one embodiment, values from a user's user measurement channels arecollected by using one or more sensors and measurement devices that maybe attached to the body, clothing (e.g., gloves, shirts, helmets),implanted in the user's body, and/or remote sensors external to theuser's body (e.g., camera, microphone).

In one embodiment, some of the user measurement channels are stored in adatabase as time series with short durations between consecutivemeasurement points. Optionally, the user's measurement channels arestored at different temporal resolutions, i.e., the typical differencein time between consecutive entries in the database may vary betweenmeasurement channels. Optionally, the temporal resolution of the samechannel may vary at different points in database, for example, regionswhere low complexity in the measurement values is detected and/orregions with noisy measurements, may be stored with lower temporalresolution, i.e., longer intervals between values. Optionally, thesystem supports queries that provide the values of the user measurementsat a required time, for example by interpolating values from the storedmeasurements at different times.

In one embodiment, the user measurement data may be processed and/ornormalized in many ways, before, during and/or after the data is stored.In one example, the values of some of the measurements are scaled to bein the range [−1,+1]. In one example, the values of some of themeasurements are normalized to z-values, which bring the mean of thevalues recorded for the modality to 0, with a variance of 1. In anotherexample, some user measurements may be processed and/or converted toanalyzable features in several ways. For example, through extractingstatistics for the values of each measurement channel in a predefinedwindow size, such as the minimum, maximum, and/or various moments of thedistribution, such as the mean, variance, or skewness. In still anotherexample, user measurements are subjected to feature extraction and/orreduction techniques, such as Fisher projections, Principal ComponentAnalysis (PCA), and/or feature selection techniques like SequentialForward Selection (SFS) or Sequential Backward Selection (SBS). In stillanother example, some of the images and video images may be processedusing various detection algorithms for identifying cues like movement,smiling, laughter, concentration, body posture, and/or gaze. The imagesmay also be processed with algorithms for detecting and describing localfeatures such as Scale-Invariant Feature Transform (SIFT), Speeded UpRobust Features (SURF), and/or scale-space representation. In stillanother example, auditory and/or written data are processed using speechanalysis and/or semantic analysis methods.

Some embodiments may utilize known and to be discovered methods forpre-processing user measurement data and extracting features from themeasured data. For example: (i) a variety of physiological measurementsmay be preprocessed according to the methods and references listed invan Broek, E. L., Janssen, J. H., Zwaag, M. D., D. M. Westerink, J. H.,& Healey, J. A. (2009), Prerequisites for Affective Signal Processing(ASP), In Proceedings of the International Joint Conference onBiomedical Engineering Systems and Technologies, INSTICC Press,incorporated herein by reference; (ii) a variety of acoustic andphysiological signals may be pre-processed and have features extractedfrom them according to the methods described in the references cited inTables 2 and 4, Gunes, H., & Pantic, M. (2010), Automatic, Dimensionaland Continuous Emotion Recognition, International Journal of SyntheticEmotions, 1 (1), 68-99, incorporated herein by reference; (iii)Pre-processing of Audio and visual signals may be performed according tothe methods described in the references cited in Tables 2-4 in Zeng, Z.,Pantic, M., Roisman, G., & Huang, T. (2009), A survey of affectrecognition methods: audio, visual, and spontaneous expressions, IEEETransactions on Pattern Analysis and Machine Intelligence, 31 (1),39-58, incorporated herein by reference; and (iv) pre-processing andfeature extraction of various data sources such as images, physiologicalmeasurements, voice recordings, and text based-features, may beperformed according to the methods described in the references cited inTables 1, 2, 3, 5 in Calvo, R. A., & D'Mello, S. (2010). AffectDetection: An Interdisciplinary Review of Models, Methods, and TheirApplications. IEEE Transactions on affective computing 1(1), 18-37,incorporated herein by reference.

Tokens

In one embodiment, the system extracts/receives/accesses a stream oftoken instances. The token instances may be annotated using anyappropriate manual, semi-automatic, and/or automated techniques. Forexample, manual token labeling may be achieved using manual annotationor marking. In another example, the token labeling is partiallyautomated using algorithms to segment media into scenes, or segmentand/or outline objects in images. In still another example, the tokeninstances are extracted from audio-visual content and labeledautomatically utilizing known and to be discovered image and/or videosegmentation, and/or object detection algorithms. In another example,some of the token instances may be extracted from audio-visualmonitoring of the user's surroundings, for example using one or moremicrophones and/or one or more cameras on the user and/or in thesurroundings. In another example, some of the tokens may be extracted bysemantic analysis of text, uttered words, conversations, blog posts,twits, or emails; such tokens may represent specific words, phrases, orconcepts that can be derived from the content of the stimuli.

In one embodiment, a single object such as an image, a word, or a soundmay be the cause of multiple token instantiations. For example, a cuteblack puppy may instantiate the tokens “dog” and “black” (its dominantcolor) and “cute” (its general appearance). A song may instantiate thetokens “Rock 'n Roll” and “loud music”.

In one embodiment, token instances are stored as records in a database.Optionally, the token instances are stored as a time-series, whereentries in the database correspond to certain times or events and maycontain information about the tokens instantiated at that time or event.Optionally, the records in the database are stored in a structure thatlinks between the token instances and emotional state annotations of theuser for times in temporal proximity to the exposure to said tokeninstances. Optionally, the records in the database are stored in astructure that links between the token instances and user measurementchannel data for times in temporal proximity to the exposure to saidtoken instances.

In one embodiment, token instances may include values for variousattributes such as a token identification number, weight, size, and/orintensity, when applicable to the type of token instance being stored.Optionally, the token instances may include the duration, and/or starttime and duration, and/or start time and end time, and/or any otherequivalent notation designating the period of time or events in whichthe user was exposed to the token instances.

In one embodiment, a token may be instantiated multiple times,optionally, at overlapping times. For example, a scene in which thereare several characters appearing, may annotated as having severalinstances of the token “person”.

In one embodiment, some tokens may be grouped into different classes,types and/or abstraction levels. Optionally, a token may belong to oneor more groups of tokens. In one example dealing with media a user isviewing, the type of media the user is watching may be a high-leveltoken group called “media type”, which may include various tokens like“movie”, “tv program”, “web cast”. On a lower level, there may be tokensdescribing scenes in a movie, which may grouped together under the label“scene type”, which may include tokens like “romantic scene”, “actionsequence”, “dramatic climax”. In another example dealing with a user'svisit to a supermarket while using a device with augmented realitycapabilities (like a smartphone), a high level token group may be“locations”, which may include tokens like “user's home”, “supermarket”,“user's office”. A lower level group of tokens may be labeled “sublocations”, and include various tokens like “dairy department”, “softdrinks section”, “checkout line”. A group of low-level tokens mayinclude specific products like “cranberry juice”, “yogurt”, “bread”.

In one embodiment, tokens may be grouped according to various criteriasuch as the tokens' typical context, and/or location of experience bythe user. In one example, a high-level token group may be “activitytype” which will typically include activities that may last hours like“watching a movie”, “rock climbing”, “reading a book”, “surfing theweb”. A low-level token group may be “images on computer screen”, whichwill include various images seen on the computer screen with a typicalshort duration.

In one embodiment, token instances may be grouped according to theirsource or cause of instantiation. For example, all token instancesinstantiated by the playing of a movie (a token for the whole movie,tokens for types of scenes, tokens for images, sounds), can be groupedas having the movie as a source. In another example, all token instancescorresponding to words and phrases appearing on a web-page share theweb-pages URL as their source, and may be grouped together accordingly.

In one embodiment, tokens may be described using one or morehierarchies. For example, a dog may have the following hierarchicaltokens: level 1—animal, level 2—dog, level 3—puppy, level 4—Labradorpuppy, level 5—black Labrador puppy. A song may be given the followinghierarchical tokens: level 1—music, level 2—rock n'roll, level 3—musicby Kiss, level 4—“Rock n'Roll all Night” performed by Kiss.

In one embodiment, patterns or subsets of tokens may be grouped togetherand represented by a new pattern token. For example, if in a certaintime interval, such as the one defined by a temporal token instancewindow, instances of the individual tokens comprising the pattern arefound, they may be replaced with the corresponding pattern token.Optionally, the pattern token's weight at that time point may equal thesum of its individual tokens' weights and/or the cardinality they havefor the purpose of token counts, for instance if saturation is includedin the model, may equal the number of tokens in the pattern. Followingthis stage, the model creation, optimization, and analysis may treat theinstances of pattern tokens as regular token instances.

In one embodiment, subsets of tokens that may serve as pattern tokensmay be found using algorithms for finding frequent patterns. Optionally,some patterns may involve attribute values of some of the tokeninstances. For example, some of the algorithms described in Han, J.,Cheng, H., Xin, D., & Yan, X. (2007), Frequent pattern mining: currentstatus and future directions, Data Mining and Knowledge Discovery,15(1), 55-86, incorporated herein by reference, may be used fordetecting frequent patterns in various ways.

User Interest/Attention

In one embodiment, some of the token instances may be assigned valuesreflecting the level of interest the user is predicted to have in saidtoken instances. The terms “interest level” and “attention level” areused herein interchangeably. Optionally, interest level data in tokensmay be compiled from one or more sources, such as (i) attention levelmonitoring, (ii) prediction algorithms for interest levels, and/or (iii)using external sources of information on interest levels. Optionally,interest level data in tokens may be stored as a numerical attribute fortoken instances. Optionally, the interest level data in tokens mayexpress the relative interest levels in the various token instances.Optionally, interest level data in tokens may be grouped into broadcategories, for example, the visual tokens may be grouped into threecategories according to the attention they are given by the user: (i)full attention, (ii) partial/background attention, (iii) low/noattention.

In one embodiment, the user's level of interests in some of the tokensmay be derived from the user measurement channels, which are processedto detect the level at which the user is paying attention to some of thetoken instances at some of the times.

In one embodiment, the general attention level may be measured, forexample by a camera and software that determines if the user's eyes areopen and looking in the direction of the visual stimuli, and/or byphysiological measurements that may include one or more of thefollowing: heart-rate, electromyography (frequency of muscle tension),or electroencephalography (rest/sleep brainwave patterns), which may beused to determine the level of the user's coconsciousness and/oralertness at a given moment. In one example, the fact that a user islooking or not looking at a display is used to determine the user'slevel of interest in a program appearing on the display.

In one embodiment, object-specific attention level may be measured forexample by one or more cameras and software that performs eye-trackingand/or gaze monitoring to detect what regions of a display, or region ofan object, or physical element the user is focusing his/her attentionat. The eye-tracking/gaze information can be compared to objectannotation of the picture/scene the user is looking at to assign weightsand/or attention levels to specific token instances, which represent theobjects the user is looking at.

In one embodiment, various methods and models for predicting the user'sinterest level are used in order to assign interest level scores forsome token instances.

In one embodiment, user interest levels in image-based token instancesare predicted according to automatic importance predicting algorithms,such as the one described in Spain, M. & Perona, P. (2011), Measuringand Predicting Object Importance, International Journal of ComputerVision, 91 (1). pp. 59-76. Optionally, the predicted level of interestfrom this type of model may be stored as an attribute value for sometoken instances. In one example, a model for predicting the user'sinterest level in various visual objects is created automaticallyaccording to the method described in Spain et al. (2011), using tokeninstances for which there is user attention-monitoring, as trainingdata.

Analysis of previous observations of the user's interest in some tokensmay be used to determine interest in new previously unobserved tokens.In one embodiment, a machine learning algorithm is used to create amodel for predicting the user's interest in tokens possibly for whichthere is no previous information, using the following steps: (i)extracting features for each token instance, for example describing thesize, duration, color, subject of visual objects; (ii) using theattention-level monitoring data as a score for the user's interest;(iii) training a predictor on this data with a machine learningalgorithm, such as neural networks or support vector machines forregression; and (iv) using the trained predictor to predict interestlevels in instance of other (possibly previously unseen) tokens.

In one embodiment, analysis of previous observations of the user may beused to determine interest in specific tokens. For example, the factthat the user has watched in the past many programs about dogs, may beused to infer his/her interest in objects and/or tokens that have to dowith dogs, or the fact a user has not missed any episode of a certainseries in the past season, can be used to infer that he/she isinterested in that certain program, and is likely to be paying attentionto the content.

In one embodiment, information gathered from other users who essentiallyexposed to the same token instances as the user may be used to assigninterest levels for the user, for example, in cases where the user'sinterest level data is missing or unreliable. In one example, whenassigning interest level to tokens extracted from a movie, at times whenthe user's eye-tracking information is inconclusive for a tokeninstance, the interest levels for that token instance can be set toaverage interest levels given to that token instance by other users whowatched the same movie.

In one embodiment, a predictor for the level of attention a user isexpected to pay to different token instances is created by combining theattention predictor models and/or prediction data from other usersthrough a machine learning collaborative filtering approach.

In one embodiment, an external source may provide the system with dataon the user's interest level in some tokens and/or token instances. Inone example, information on users interest may be provided by one ormore humans by answering a questionnaire indicating current areas ofinterest. The questionnaire may include areas such as: pets,celebrities, gadgets, media such as music and/or movies (genres,performers, etc.), and more. The questionnaire may be answered by theuser, friends, relations, and/or a third party. In another example,semantic analysis of the user's communications such as voice and/orvideo conversations, instant messages, emails, blog posts, twits,comments in forums, keywords use in web searches, and/or browsinghistory may be used to infer interest in tokens describing specificsubjects, programs, and or objects of interest. In yet another example,some of the user's subjects of interest may be provided bythird-parties, such as social-networking sites like Facebook, and/oronline retailers like Amazon.

In one embodiment, a temporal attention level is computed for the userat a specific time. Optionally, the temporal attention level is storedas a time series, for example, at each time point the system records thetemporal attention level score of the user on a scale in the range[0,1], where 0 indicates that no attention being paid, and 1 indicatesthat full attention is being paid. Optionally, temporal attention leveldata may be extracted from a visual attention data source (e.g.,eye-tracking, face expression analysis, posture analysis), an auditorydata sources, monitoring the users movement, and/or physiologicalmeasurements (e.g., EEG).

In one embodiment, interest levels obtained from various sources arecombined into a single “combined interest level score”. The combinedinterest level score may be stored as an attribute in some of the tokeninstances. In one example, the interest level scores from varioussources such as attention-level monitoring, predicted interest based onthe user's historical attention-levels, and/or interest data receivedfrom external data sources, may be available for a token instance. Eachinterest level sore is provided as a value in the range [0,1].Optionally, the combined interest level score may be a weightedcombination of the values from the different sources, where each sourcehas a predefined weight.

Weighting and Normalizing Token Instance Weights

In one embodiment, token instances are given a weight attribute, whichis correlated with the magnitude of the token instances' influence onthe user's affective response. Optionally, a token instance may have asingle or multiple values for the weight attribute. For example,multiple values may describe the token instance weight at various timepoints.

In one embodiment, the system supports queries that provide the valuesof a token instance's weight at a required time, for example byinterpolating values from token instance weights at different timepoints.

In one embodiment, the token instance weight is a pre-determined value.In one embodiment, q weight attributes are assigned to a token instance,for example, weights w₁, . . . , w_(q) for q different segments of theduration of the token instance's existence. The total weight assigned tothe token instance equals w=w₁+ . . . +w_(q), and may be distributed tothe q attribute values in different ways. In one example the weight isdistributed uniformly, such that w₁= . . . =w_(q)=w/q. In anotherexample, the weights may be distributed in a non-uniform way. Forexample, by assigning higher weights to earlier segments in the durationof the token instance's existence to reflect the fact that the token'sinfluence on the affective state diminishes as time goes by. Optionally,the weight assignment to various points may follow a parametricdistribution, such as an exponential or Gamma distribution, withpredefined parameters, and/or parameters that are set as part of theaffective response model training.

In one embodiment, attribute values for some of the token instances maybe used to modify the weights of the token instances. For example,tokens that have an attribute “size” or “intensity” may be reweighted,for instance, by multiplying the token instance weight with theattribute value(s), to reflect the fact that instances that are largeror more intense have a stronger influence on the user's affective state.

In one embodiment, some of the interest level scores may be used toreweight token instances. For example, the token instance weight may bemultiplied by the attention score for the token instance which is in therange [0,1], or multiplied by the average attention score.

In one embodiment, different types of tokens may be assigned differentattention levels at the same time, depending on the circumstances. Forexample, when viewing media on a screen, if it is determined from acamera monitoring the user that he/she is momentarily not looking at thescreen, all visual token instances may be given an interest levelreflecting that fact, for instance a weight of 0. At the same time, itmay be assumed that the user is still listening, therefore,sound-related token instances may still be given a weight greater thanzero.

In one embodiment, general attention levels are allowed to affectlong-lasting token instances that involve things like the genre,program, or main characters. Short lasting token instances, like objecttokens such as a dog or a car, should not be influenced by the generalattention levels unless the content is about the specific tokeninstance. For example, a user is watching a movie where the scene takesplace in a living room. If the user is not paying attention to themedia, there is no reason to assign a weight to token instances of itemsin the background of the scene, such as the sofa or napping dog, sincethe user is not paying attention to the scene and thus anything learnedabout the user's affective response towards those token instances isprobably noise.

In one embodiment, the weights of token instances in a database arenormalized. For example, the weights may be normalized in such a waythat the sum of weights from all token instances in the database at agiven time, or all instances in a temporal token instance window, equala constant. Optionally, weights of token instances may be normalized insuch a way that depends on the sum of the token instance weights(denoted by S). For example, token instance weights may be normalized insuch a way that they sum up to log(1+S), or the square root of S.

In one embodiment, some token instance weights are reweighted and/ornormalized before the affective response model is trained. In oneembodiment, some token instance weights are reweighted and/or normalizedwhile the affective response model is trained.

Representing Emotions

In one embodiment, the user's emotional state is annotated at some timepoints, or for some temporal token instance windows, using variousmethods for representing emotions. Optionally, the annotations areobtained utilizing a transformation from a domain representingmeasurements to a domain representing internal emotional states.Optionally, the user's emotional state is annotated by the user. Forexample, the user's emotional state may be represented in one of thefollowing methods.

In one embodiment, emotional states are represented using discretecategories. For example, the emotion categories may include threecategories: negatively excited, positively excited, and neutral. Inanother example, the emotion categories include happiness, surprise,anger, fear, disgust, and sadness.

In one embodiment, emotional states are represented using amultidimensional representation, which characterizes the emotional statein terms of a small number of latent dimensions. In one example, theemotional states are represented as points in a two dimensional space ofArousal and Valence. Arousal describes the physical activation andvalence the pleasantness or hedonic value. Each detectable experiencedemotion is assumed to fall in a specified region in that 2D space. Otherdimensions that are typically used to represent emotions include:potency/control (refers to the individual's sense of power or controlover the eliciting event), expectation (the degree of anticipating orbeing taken unaware), and intensity (how far a person is away from astate of pure, cool rationality). The various dimensions used torepresent emotions are often correlated. For example, the values ofarousal and valence are often correlated, with very few emotionaldisplays being recorded with high arousal and neutral valence. In oneembodiment, emotional states are represented as points on a circle in atwo dimensional space pleasure and arousal.

In one embodiment, emotional states are represented using a numericalvalue that represents the intensity of the affective state with respectto a specific emotion. For example, a numerical value stating how muchthe user is enthusiastic or happy. Optionally, the numeric value for theemotional state may be derived from a multidimensional spacerepresentation. For example, let P be a path or collection of points inthe multidimensional space. For every point p in P, a numerical valued(p) can be computed, for instance by computing the distance that needsto be traveled along P from a reference point serving as zero to reachp. Given a point q in the multidimensional emotional space, which is notin the set of points P, the projection of q on P is a point q′ in P, forwhich the Euclidean distance (q,q′) is minimal. The numerical value forthe point q may be assigned the same value as its projection on P, whichequals d(q′).

In one embodiment, emotional states may be modeled using componentialmodels that are based on the appraisal theory, as described by the OCCmodel (Ortony, Clore & Collins, 1998). According to this theory, aperson's emotions are derived by appraising the current situation(including events, agents, and objects) with respect to the person goalsand preferences.

In one embodiment, emotional states represented by categories areconverted to a multidimensional representation. For example, this can bedone by assigning each category a representative point in themultidimensional space.

In one embodiment, emotional states described as points in amultidimensional space are converted into a categorical representationin several ways. In one example, there are predefined categories, witheach category having one or more representative points in themultidimensional space. An unassigned point P in the multidimensionalspace may be assigned to the category that has a representative point P′for which the Euclidian distance between P and P′ is smaller or equal tothe distance between P and all other category representative points. Inanother example, the multidimensional space representing emotions ispartitioned into a number of regions that cover the entiremultidimensional space. Following that, the points falling in the sameregion belong to the same category. For example, a valid partition maybe splitting the arousal-valence space into four quadrants;consequently, each point in the multidimensional space belongs to one ofthe four resulting categories. In yet another example, themultidimensional emotional space contains N or more points describingemotions, for example, from the emotional states of multiple peopleand/or multiple time points. The points in the multidimensional spaceare then clustered into N clusters, using an algorithm like K-means withthe Euclidean distance metric. Each cluster may then represent acategory, with the mean of each cluster serving as a representativepoint for the category. Each existing or new point P in the dimensionalspace may then be assigned a category by choosing the category whoserepresentative point has a minimal distance to P.

In one embodiment, emotional states are described using a scalar value,and may be converted to a categorical representation. For example, usingpredefined categories, where each category has one or morerepresentative scalar value(s). An unassigned point P in the space maybe assigned to the category which has a representative point P′ forwhich the value |P−P′| is minimal.

In one embodiment, a method is provided for computing the distancebetween two emotional state annotations. In one example, the distancebetween two categorical annotations may be computed using a pre-defineddistance matrix that holds the distance between pairs of annotations. Inanother example, the distance between two annotations in a scalarrepresentation may be equal the absolute value of the result of thesubtraction of the value of the first annotation from the second. In yetanother example, the distance between two annotations using amultidimensional emotional state representation may equal the Euclideandistance between both annotations.

Annotations

In one embodiment, machine learning algorithms are trained on dataextracted from user measurement channels in order to create a model forpredicting a user's emotional state at a required point in time.Optionally, data comprising token instances may also be used fortraining these models.

In one embodiment, models for a user's emotional state are periodicallyre-trained and/or updated to reflect new data that has been accumulated.Optionally, the models are re-trained following an event where theprediction error exceeds a threshold, and/or following an event wherethe performance deteriorates below a threshold.

In one embodiment, the data extracted from the user measurements may benormalized with respect to the user's baseline for that time.Optionally, the normalization is performed periodically, such as everyfew hours or every day. Optionally, the normalization is performedfollowing a large change in one or more of the user measurementchannels, such as resulting from a situation change.

In one embodiment, a baseline function for the annotated emotional statemay be used as an input to a machine learning algorithm for predictingthe user's emotional state.

Some embodiments may utilize known and to be discovered systems topredict the emotional state from single or multiple user measurementchannels. The predictions may use various methods for emotionalrepresentation, such as categorical, dimensional, and/orappraisal-based. Examples of emotional state prediction methods that maybe use include: (i) physiological-based predictors as described in Table2 in van den Broek et al. (2009); (ii) Audio- and visual-basedpredictors as described in Tables 2-4 in Zeng, Z., et al. (2009); (iii)additional predictors for the emotional state that are bothsingle-channel (unimodal) or multi-channel (multimodal) as described inTables 2, and 4 in (Gunes & Pantic, 2010); and/or (iv) predictors of theemotional state from low-level media features, such as described inHanjalic, A., & Xu, L.-Q. (2005). Affective video content representationand modeling. IEEE Transactions on Multimedia, 7(1), 143-154.

In one embodiment, the machine learning system for predicting the user'semotional state may need to make decisions from multiple usermeasurement channels. Therefore, at some stage, the data from thedifferent user measurement channels may need to be fused. Optionally,different types of data fusion may be employed, for examplefeature-level fusion, decision-level fusion or model-level fusion, asdiscussed in Nicolaou, M. A., Gunes, H., & Pantic, M. (2011) ContinuousPrediction of Spontaneous Affect from Multiple Cues and Modalities inValence-Arousal Space, IEEE Transactions on Affective Computing.

In one embodiment, the user's emotional state at certain time points islabeled by monitoring the user. In one example, the user is presentedwith sensual stimuli that are known to predict certain emotions, such asimages, videos, and/or sounds that are known to elicit an emotionalresponse. In another example, the user is presented with media clipswhich he views and after each one reports the elicited emotions (e.g.,positive, negative, or neutral), or is asked to provide values in adimensional space, for example in the Arousal/Valence dimensions.Optionally, users may use systems that aid emotional state annotation,for example, a system that describes various emotional states usingcartoon images. In yet another example, the user may have the option tocreate training samples, for instance, by indicating that what he/shejust saw gave a good or bad feeling. Alternatively, the user may beasked to imagine scenarios in which certain emotions are felt, and theuser measurements at that time may be used as training sample.

In one embodiment, a pre-trained model for predicting the emotionalstate from user channel measurements is used to label the user'semotional state at certain time points. Optionally, the system istrained on data from multiple users.

In one embodiment, a pool of models for predicting the emotional statefrom user channel measurements may be available to label the user'semotional state. Optionally, each model was trained using a singleindividual. In order to accurately label the user's emotional state, amodel belonging to a person similar to the user is selected for thelabeling process. The similarity between people may be determined inseveral ways, for example, by observing similar patterns in the valuesof their user measurement channels and/or token instances to which theywere exposed, by observing similar demographic and/or educationalcharacteristics, and/or by semantic analysis of speech, text, and/orvideo content created by the people determine similar attitudes and/orworld views.

In one embodiment, one or more methods are used to label an initial setof training points with the user's emotional state. These labeledpoints, in turn, are used to train the user's model for predicting theemotional state. Following that, several rounds of bootstrapping mayensue, in which the user's model is used to label additional points,which are then used to retrain the user's model. With each iteration,the user's model may better fit the training data. Optionally, themodel's performance is tested on an independent test set, which waslabeled using a different model (for example trained on another portionof the data set), in order to prevent over-fitting. Optionally, othersemi-supervised training methods may be used to create the model forpredicting a user's emotional state.

Situations

In one embodiment, a user's exposure to tokens and/or the user'sresponse are monitored over a long period of time that spans varioussituations. Different situations may affect the user's behavioral andresponse. For example, the user's reaction to certain token instancesmay change depending on the situation in which the user was in whilebeing exposed to the token instances. For example, a user may reactcalmly to adult-themed material when viewing alone (one situation),however the same user may react angrily if such material was to appearwhile the user's children were present (a different situation). Inanother example, the user's baseline value for an emotional state and/oruser measurement channel may change significantly in differentsituations, such as when the user is driving vs. relaxing at home, orwhen the user is alert vs. inebriated.

In one embodiment, information describing a user's situation duringcertain times and/or events is stored in a database. Optionally, somesituations are described using one or more token instances and/orattribute values of one or more token instances. Optionally, informationdescribing a situation is linked to some of the token instances.Optionally, information describing a situation may be linked to databaserecords corresponding to temporal token instance windows. Optionally,the information describing some situations may be in the form ofsituation identifiers.

In some embodiment, situation identifiers are received describing theuser's situation during certain times and/or events. Optionally, aclassifier may be trained to identify the situation occurring during newtimes and/or events. Optionally, the training samples used to train sucha classifier comprised of one or more of the following elementscorresponding to a certain time and/or event: values of some tokeninstances and/or their attributes, values from one or more usermeasurement channels, an emotional state annotation, a baseline valuefor the emotional state, and/or baseline values for one or more usermeasurement channels. Optionally, some training samples are assignedlabels corresponding to their respective situation identifiers.Optionally, a machine learning classification algorithm is trained usingthe training samples, in order to produce a classifier that may identifythe situation in which the user is at various times and/or events. Forexample, the machine learning algorithm used to train a classifier maybe a neural network classifier, a support vector machine, maximumentropy classifier, or a random forest. In one embodiment, the trainingdata may be comprised of labeled and unlabeled data (for which thesituation is unknown), and a semi-supervised machine learning method maybe employed to train the classifier.

In one embodiment, the data may initially not include a description ofsituations, so situations may need to be discovered from the data in anunsupervised, or semi-supervised fashion. The following examplesdescribe different situation characterizations to be discoveredautomatically:

(i) Certain situations may be characterized by a small set of specifictoken instances. For instance, the situation in which the user isdriving may be characterized by a token instance “user is driving”,while the situation that the user is not alone may be characterized by atoken instance “other people are in the user's vicinity”.

(ii) Different situations may be characterized by the fact that the useris exposed to a different characteristic set of similar token instances,and/or tokens originating from similar sources. For example, in asituation labeled “watching TV” a user may be exposed to visual tokensbelonging to images, while in a situation labeled “reading”, the usermay be exposed to many tokens corresponding to words.

(iii) Some situations may be characterized by the fact that the userexhibits a noticeably different response to instances of certain tokens.For example, in a situation that may be labeled “hungry” the user mayexperience a certain affective response such as high arousal whenexposed to tokens related to food. However, in a situation labeled “nothungry”, the user may exhibit a much milder affective response to tokeninstances related to food.

(iv) Some situations may be characterized by the fact that the user hastypical baseline levels for the affective state and/or one or more ofthe user measurement channels. For example, when in the situation“driving” a user may have a typical affective state baseline that is ata high level of arousal for the whole duration or most of time in whichthe user this situation, while when the user is in the situation“relaxing at home”, the baseline arousal level is much lower.

(v) Situations may be characterized by the fact that samplesrepresenting certain times or events in which the user was in the samesituation, tend to belong to the same clusters when an unsupervisedclustering algorithm is run on the samples. For example, the samples maybe comprised of vector representations of temporal token instancewindows. When a clustering algorithm like K-means is applied to thedata, each cluster of vectors of windows may be considered a differentsituation.

In one embodiment, a method for detecting different situations utilizesa mixture model and an Expectation Maximization (EM) algorithm.Optionally, the method comprises the following steps:

(i) Acquiring or receiving samples—data collected for certain times orevents is received, for example in the form of vector representations oftemporal token instance windows. Optionally, variables in the vectorsparticipating in the EM likelihood computation may be assumed to comefrom a distribution with a parametric form, for example, a variable withk possible values may be considered to have a discrete distribution withk parameters p₁, . . . , p_(k), such that p_(i)≧0, for i=1 . . . k, andp₁+ . . . +p_(k)=1, which correspond to the variable's k possiblevalues. In another example, a real-valued variable may be assumed tohave a parametric distribution, such as a Gaussian with parameters μ,σ².

(ii) Initializing—The method receives a desired number of situations N.Initially some of the samples are assigned to the N situations.Optionally, some situation assignments are received as an input.Optionally, the samples are randomly assigned situations to Nsituations. Optionally, an initial partitioning of samples is done usingone or more of the aforementioned characterizations of differentsituations. For example, all samples with a certain token instance(e.g., “driving”) may be assigned the same situation. In anotherexample, all samples for which there is a similar user measurementchannel value (such as low heart-rate), are assigned the same situation.Optionally, the assignment process to situations may be hierarchical,and use more than one round of partitioning. For example, all samplegrouped together because of a low heart-rate may be further refined byclustering them into several clusters. The partitioning into situationsmay continue until the total number of groups of samples (correspondingto situations) reaches N. Optionally, certain samples may belong to morethan one situation (i.e., their weight may be spread over severalsituations).

(iii) Training a model using and EM approach—looping until the modelconverges, performing the following steps:

Expectation—Using the current set of parameter values, compute for eachsample the probability that it belongs to each of the situations.Optionally, in the first iteration, the initial assignments to situationgroups may be used instead.

Maximization—Re-estimate the parameters using maximum a posterioriparameter estimation to find a set of parameter values that maximizesthe expected log likelihood of the data.

(iv) Returning results—returning the model's parameters afterconvergence and/or situation assignments for samples in the form ofprobabilities of a sample being in the N different situations orreturning only the most likely situation for each sample.

In one embodiment, samples with situation assignments obtained with theEM procedure may be used to train a classifier for predicting thesituation for new unseen samples. Optionally, a machine learning methodsuch as a neural network, support vector machine, random forest,decision tree, or other machine learning classification algorithms, maybe used for this task.

In one embodiment, a method for identifying different situationsutilizes a clustering algorithm. Data collected for certain times orevents is received and converted to samples that may be processed by theclustering algorithm, for example in the form of vector representationsof temporal token instance windows. Optionally, the clustering algorithmuses a distance function for samples that utilizes a function forcomputing the distance between two temporal token instance windows.Optionally, the clustering algorithm utilizes a distance function thatcomputes the distance between samples' emotional state annotations.Optionally, the distance function between samples combines the twomeasures, for example using a weighted sum, where a predefined weightα>0 is given to distance between temporal token instance windows of thetwo samples, and a weight 1−α for the distance between the emotionalstate annotations of the two samples.

In one embodiment, a hierarchical bottom-up clustering algorithm is usedto identify situations. Given a pre-defined number of desiredsituations, the algorithm is run iteratively, starting off with eachsample as a singleton cluster, and in each iteration joining the mostsimilar clusters. Optionally, the algorithm proceeds to run until athreshold is reached such as a maximal distance between joined clusters.Optionally, the algorithm proceeds until the number of clusters reachesa predefined target.

In another embodiment, partitioning clustering algorithm such as K-meansis used to identify situations. Optionally, the algorithm is run using apre-specified number of desired clusters. Optionally, the clusteringalgorithm is run several times with a different number of desiredclusters, choosing the configuration giving a desired tradeoff betweenthe number of clusters and the tightness of clusters (expressed forexample, as mean squared distance to the cluster centers).

In one embodiment, after running the clustering algorithm, each sampleis assigned a situation identifier corresponding to its cluster. Thesamples with situation assignments may be used to train a classifier forpredicting the situation for new unseen samples. Optionally, a machinelearning method such as a neural network, support vector machine, randomforest, decision tree, or other machine learning classificationalgorithms, may be used for this task.

Baselines

In one embodiment, one or more baseline levels (also referred to in thisdisclosure as “baseline values”) are computed for the user's responseindicators, which may be the user's affective state and/or values ofsome of the user measurement channels. A user's baseline level for aresponse indicator is a representative value of the user's usual state,computed from multiple values of the response indicator acquired over along period, such as a few hours, a day, a month, or even a year. Abaseline level usually reflects the expected value for the responseindicator when not considering the effects of the user's shortterm-exposure to token instances.

In one embodiment, a user's baseline level is computed for a categoricalresponse indicator, such as the user's emotional state, which isrepresented by emotional categories. Optionally, the baseline level iscomputed by observing the values of the categorical response indicatorover a long period, and using for a baseline level the category that wasthe response indicator's value the longest time. Optionally, thebaseline level comprises a set of values that describe the proportion oftime the response indicator had each of the categorical values.

In one embodiment, a user's baseline level is computed for a real-valuedresponse indicator, such as a user measurement channel or a dimension inan emotional state representation. Optionally, the baseline level iscomputed from observations of the response indicator's values collectedover a long period. Optionally, a baseline level may bemultidimensional, being comprised of several baselines corresponding toindividual dimensions. For example, a user's baseline may be comprisedof a baseline level for the user's arousal and a baseline level for theuser's valence.

In one embodiment, a user's baseline level for a response indicator iscomputed using a window of a fixed duration (such as an hour, a day, aweek), from which the values of the response indicator are collected forthe baseline computation. Optionally, the baseline level is computedwhen the user is in a specific situation (such as being alone, watchingtv, being in a happy mood) by collecting values of the responseindicator during periods when the user was in essentially the samesituations.

In one embodiment, a user's baseline level for a response indicator iscomputed by collecting multiple values of the response indicator,optionally over similar situations, and applying various computationalprocedures to the collected values, such as: (i) averaging values in asliding time window of a predefined size; (ii) a weighted average of thecollected values; (iii) low-pass filtering to the values; (iv) Fouriertransform to the collected values; and/or (v) wavelet transform analysisto the data.

In one embodiment, a baseline value may be comprised of a weightedcombination of several baseline values computed from data collected atdifferent time-scales and/or situations. For example, a baseline for theemotional state of a user watching an action movie on a television setmay be comprised of the following baselines: 20% of the weight is givento the user's baseline computed from data collected during the previous24 hour period (in all situations), 30% of the weight is given to thebaseline computed from data collected from the user's most recent twohours of television viewing (of any program type), and the remaining 50%of the baseline weight is given to the baseline computed from the last100 hours of the users viewing of action content (such as movies ortelevision programs).

In one embodiment, the user's baseline level values for a responseindicator computed at different times and/or situations are stored in adatabase. Optionally, additional values are stored in the database suchas (i): values and/or baseline values for user measurement channels;(ii) values and/or baseline values for the user's emotional state; (iii)situation identifiers denoting situations the user was in during theperiod in which data was collected for the baseline computation; (iv)values and/or baseline values of tokens describing the user's situation;and/or (v) linkage information between baseline values at certain timesand records corresponding to temporal token instance windows.

In practice, the ideal baseline function is often not a simple smoothaverage function, because the baseline level may dramatically depend onthe user's situation. As a result, the user's baseline level may changesignificantly during a short duration of time when there is asignificant change in the user's situation. Thus in practice, thebaseline does not resemble a smooth slow-changing function typicallyobserved when the baseline is computed as an average of values collectedover long periods of time. For example, when the user is alone at homewatching TV, he/she has one baseline level for the emotional state; ifthe user's mother-in-law enters the house, the user's emotional statebaseline may change significantly in a very short time, and remain inthat changed state for the duration of the mother-in-law's presence.This rapid change in baseline may not be reflected in a timely manner ifthe baseline is computed simply by averaging values in a large temporalwindow, while disregarding the context of different situations.

In one embodiment, a user's baseline level is predicted using a machinelearning method, such as a support vector machine, a regression method,a neural network, or support vector machine for regression. The trainingdata for the machine learning method may include samples comprisingresponse indicator values and various input variable values. Optionally,the data for the samples is collected while the user is in specificsituations, in order to train situation-specific baseline predictors forthe user.

In one embodiment, the training data for a machine learning-basedbaseline value predictor for a user comprises data of the followingtypes:

(i) Computed baseline values for the user for the response indicatorand/or other variables (such as user measurement channels). Optionally,the baseline values are computed using data collected in different ways,such as by collecting values from time intervals of different durationsand/or times in which the user was in certain situations. Optionally,the baseline values for the time windows computed for the data from thetime intervals using various methods such as averaging, low-passfiltering, Fourier Transform, and/or wavelet transform.

(ii) Situation identifiers and/or values of some token instances, ortheir attributes, at specific times (such as the time for which thebaseline is predicted), which may be used to define the user'ssituation. Note that the token instance values provided to the machinelearning method typically include long-lasting token instances thatdescribe properties like the user's activity (e.g., watching a movie,driving, being massaged), or properties describing the user'senvironment (e.g., at home, sitting alone), or the user's state or mood(e.g., excited, tired). Such long-lasting token instances may havelong-lasting influence on the user's baseline values.

(iii) Baseline values computed or collected from other data sources,such as models of other users.

In one embodiment, after using a machine learning training method toprocess the training data, the resulting model is used to predict theuser's baseline level for a response indicator. When informationregarding the user's situation and/or situation-specific inputs areprovided (such as baselines for specific situations), the resultingpredictions for baseline values made by the classifier may rapidlyadjust to situation changes that may lead to dramatic changes in theuser's baseline level for some of the variables.

In one embodiment, a method for calculating a situation-dependentrapidly-adjusting baseline value predictor for a user, utilizes adatabase, in which there are annotations describing the user's response,and corresponding situation identifiers, for at least some of theannotations. Optionally, some of the annotations stored in the databasecorrespond to emotional states. Optionally, some of the annotationsstored in the database correspond to values from the user's measurementchannels.

In one embodiment, a database is provided with a situation identifier,and optionally, a time scope from which to extract annotations. Thedatabase retrieves annotations corresponding to the provided situationidentifier, optionally, occurring within the provided time scope. Insome examples, the retrieved annotations span along time, in which theuser may have been in various situations. Thus, the retrievedannotations may correspond to intermittent time intervals, separated byintervals of time in which the user was in situations other than the onedesignated by the provided situation identifier.

In one embodiment, a baseline value for the response indicator iscomputed from the retrieved annotations, for example, by averaging thevalues, or performing low-pass filtering on the values. Since the valuesretrieved from the database correspond to a specific situation, thecomputed baseline also corresponds to the situation. Thus, even if theuser changes situations in a very short time, the computed baseline isadjusted rapidly to reflect the new situation. These situation-dependentchanges in the baseline level occur despite the fact that the baselinemay be computed by a weighted average data spanning a long duration.This is opposed to a sliding window approach in which, typically, if thewindow has a length of T (for example T=1 hour), the computed averagechanges slowly, taking time to reflect situation-dependent changes inthe baseline. Thus, even after a situation changes, it typically takes asliding window a long time, such as T/2, to significantly reflect thechange in the baseline.

In one embodiment, a method for calculating rapidly-adjusting baselinevalue for a user, utilizes a database which comprises data such asannotations describing the user's affective response, informationregarding the token instances the user was exposed to and/or values fromthe user's measurement channels. Optionally, the data in the database isaccessible as a collection of temporal token instance windows and theircorresponding annotations. In addition, a distance function forcomputing the distance between database records is also utilized, forexample, a function that computes the distance between vectorrepresentations of temporal token instance windows. The computation ofthe baseline comprises the following steps:

(i) The system is given a database an input record describing the timefor which the baseline needs to be computed;

(ii) Optionally, the system is given a time interval from which databaserecords may be samples. Optionally, if one is not provided, the systemmay use a fixed time interval such as the last 48 hours;

(iii) The system scans the database in the time interval, retrieving allrecords whose distance from the input record is below a pre-definedthreshold; And

(iv) The values from which the baseline is to be computed are extractedfrom the retrieved records, and a baseline value is computed forexample, by averaging the values, low-pass filtering or applying Fouriertransform analysis.

Windows

In one embodiment, individual temporal token instance windows may beassigned weights. For example, a certain window may be given a higherweight than others if it is deemed more important, for instance, if itappears before a time where there is significant change in the user'spredicted emotional state and/or user measurement values. In anotherexample, certain windows may cover times in which the measurements areknown to be more accurate, so the window weights may be increased inorder to increase these windows' influence during model training.

In one embodiment, the token instances in a window are represented by avector. For example, where the number of possible different tokens is N,a window is represented by a vector of length N, where position i in thevector holds the sum of the weights of all instances of token i in thewindow, or zero if there were no instances of token i in the window.Optionally, position i in the vector may hold the number of instances oftoken i that existed in the window.

In one embodiment, the vectors representing windows are preprocessedand/or modified using some of following embodiments.

In one embodiment, the set of token instances in a window may befiltered to exclude some of the token instances. For example, only thetop K token instances with the highest weights are represented in awindow.

In one embodiment, the weight values in a window's vector arenormalized. Optionally, the weights are normalized so the sum of theweight attributes in the window's vector equals a pre-defined constant,for example 1. Alternatively, the weights in the vector, which beforenormalization sum up to W, are normalized to sum up to a function of W,such as log(1+W), or the square root of W. Optionally, the tokeninstance weights are normalized according to the duration of the window.For example, by dividing a token's instance weight by the duration ofthe window, or by a function of the duration of the window.

In one embodiment, additional values may be added to a window's vectorthat are derived from various sources, such as the attribute values forthe token instances in the window, attribute values from other windows,or various baseline values.

In one embodiment, the window vectors include variables derived from thetoken instances' attribute values. For each attribute, variables may beadded in one or more of the following ways: (i) A single variablerepresenting all instances in the window. For example, the attributedescribing the general interest level, as measured by an eye-trackingdevice, may be added as a single variable. In another example, a singlevariable “intensity” may be added to the vector and given the averagevalue of the intensity attribute for all token instances that have thatattribute. (ii) Multiple variables representing different groups oftoken instances. For example, separate variables for the attribute soundenergy may be created for different types of token instances, such as“short sounds”, tokens whose source is “music videos”, and those whosesource is “classical music”. (iii) Multiple variables for each tokeninstance. Optionally, each token instance may have variables such asweight, interest level, and size.

In one embodiment, the vectors describing temporal token instancewindows include variables describing a baseline value corresponding tothe window's scope, are added to the vector.

In one embodiment, the vectors describing temporal token instancewindows include variables describing the difference between the user'sstate at a certain time (e.g., user emotional state, or a value from auser measurement channel) and the user's baseline value for thecorresponding time (e.g., the predicted baseline value for the user'semotional state or user measurement channel value).

In one embodiment, the vectors describing temporal token instancewindows include a variable describing the temporal token instancewindow's duration.

In one embodiment, the vectors describing temporal token instancewindows include variables describing the number of token instances inthe window and/or their weight. Adding such a variable may assist toincorporate the affects of saturation into models. Often when saturationoccurs, the effect of an additional stimulus is diminished when a largenumber of stimuli are experienced simultaneously. Optionally, separatevariables may be added for different groups of tokens, such as imagetokens, word tokens, or music tokens.

In one embodiment, the vectors describing temporal token instancewindows include variables describing the number of times a token hadbeen instantiated previously in various windows. For example, a variabledescribing how many times in the past minute/hour/day a token had beeninstantiated is added for some tokens or groups of tokens in order toassist models account for affects of habituation, where repeatedexposure to the same stimuli may diminish their effect on the user.

In one embodiment, variables in the vectors describing temporal tokeninstance windows may be split into b variables representing b bins forthe value of the variable, each representing a different range ofvalues.

In one embodiment, a variable may be split into several conditionalvariables, all corresponding to the same original variable; however,only one of the derivative variables is given a value in each window.The choice of which of the variables is given a value may depend on avalue from another source such as token instance or baseline value. Forexample, a variable corresponding to the token “movie” may be split totwo separate variables according to a token named “viewer is watchingalone”, so if the user is watching the movie alone, a non-zero value isgiven to one variable corresponding to “movie”. And if the user is notalone, a non-zero value is given to the other variable. In anotherexample, variables are split according to the value of a baseline forthe user. For example, splitting a variable according to the quadrant ina 2D arousal/valence space in which the baseline value falls. Suchsplits may assist certain models account for the fact that certain tokeninstances may have a dramatically different effect on the user,depending on the user's baseline emotional state (such as the personbeing in a good or bad mood).

In one embodiment, a new variable in the vectors describing temporaltoken instance windows may be assigned values that are the result of afunction applied to one of the values of one or more variables in thevector. For example, a variable may equal the square of the weightassigned to a token instance. In another example, a variable may equalthe weight of a token instance multiplied by the interest level of thattoken instance. Optionally, the new variable may replace one or more ofthe variables used to assign its value.

In one embodiment, a distance function is provided for computing thedistance between two vectors representing temporal token instancewindows. In one example, the distance function computes the dot-productof the two vectors. In another example, the distance function computesthe distance between the two vectors using a metric such as theEuclidean distance or Hamming distance. In yet another example, where X₁is the set of token instances in the first window and X₂ is the set oftoken instances in the second window, the distance between the windowsequals 1−(|X₁∩X₂|/|X₁UX₂|).

In one embodiment, a “target value” describes a state or a measurementcorresponding to a temporal window of token instances. For example, atarget value may be an emotional state prediction of the user, or avalue derived from the user measurement channels. Optionally, the targetvalue may be represented by discrete categories, a univariate value, ora point in a multidimensional space. In one example, the target valuerepresents a transition between two categorical states. In anotherexample, the target value represents the difference between the user'sstates at the end and beginning of the window. In still another example,the target value represents an average value of a variable over thescope of the window.

Databases

In one embodiment, a database stores a list of the token instancesrepresenting stimuli that may influence a user's affective state.Optionally, each token instance in the list is stored as a tokenidentifier linked to a record comprising additional attributes such asbeginning time of the token's instantiation and/or the user's exposureto the token instance, duration of exposure and/or instantiation, theweight of the token instance, the user's interest/attention level in thetoken instance. Optionally, the database also includes affectiveresponse annotations, for example, an emotional state represented as acategory, a scalar, or multidimensional value. Some of the stored tokeninstances may be linked to the annotations, for instance by storing theannotation as an attribute of the token instances. Optionally, thedatabase also includes situation identifiers, describing the user'ssituation when being exposed to some token instances. For example, byadding the situation identifiers as attributes of the token instances.Optionally, the database is used to supply data for training a machinelearning-based affective response model for the user.

In another embodiment, a database stores a collection of temporalwindows of token instances. Optionally, each window has a fixedduration, for example, ten seconds. Optionally, the token instances andsome of their optional attributes (such as weight, interest/attentionlevel, size) are represented by a vector of values. Optionally, windowsmay be assigned annotations representing affective responses, such as anemotional state and/or change in state represented as a category, singledimensional value, and/or multidimensional values. Optionally, windowsmay be assigned one or more situation identifiers, denoting the user'ssituation when being exposed to the tokens in the window. Optionally,the database is used to supply data for training a machinelearning-based affective response model for the user.

In one embodiment, the token instances stored in a database are obtainedfrom long-term monitoring of the user, for example, for a period lastingfrom days to years. Optionally, the token instances stored in thedatabase originate from multiple token sources, and the user may beexposed to them in many different situations. Optionally, the user isexposed to more than one token instance simultaneously, i.e., the useris exposed to multiple tokens with overlapping instantiation periods.Optionally, some of the stored tokens instances comprise representationsof elements extracted from digital media content, such as images,sounds, and/or text. Optionally, some of the stored tokens instancescomprise representations of elements extracted from an electromechanicaldevice in physical contact with the user.

In one embodiment, database storing information about token instances,also stores information from at least one user measurement channel.Optionally, the database includes linking information such as timestamps to associate between token instances and the user measurementchannels measured in temporal vicinity of the exposure to some of thetoken instances. Optionally, the user measurement channels may be storedat different time resolutions, for example, values of EEG signals maystored every 50 milliseconds, while skin temperature may be stored everytwo seconds.

FIG. 11 illustrates a memory 304 for storing data for access by anapplication program 302 being executed on a data processing system. Adata structure 305 stored in the memory 304 includes informationresident in a database 304 used by the application program 302. Theapplication program 302 trains a machine learning based affectiveresponse model 306 for a user, using the data structure stored in thememory. The data structure 305 stored in the memory including: tokeninstances representing stimuli that influence a user's affective state,stored in the memory. The stored token instances are spread over a longperiod of time that spans different situations, and a plurality of thestored token instances have overlapping instantiation periods. Datarepresenting levels of user attention in some of the token instancesused by the application program to improve the accuracy of the machinelearning based affective response model for the user. Annotationsrepresenting emotional states of the user, stored in the memory. Thestored annotations are spread over a long period of time that spansdifferent situations. And linkage information between the tokeninstances, the data representing levels of user attention, and theannotations. Further illustrated an optional computer-readable medium308 includes instructions, which when executed by a computer systemcauses the computer system to perform operations for training themachine learning based affective response model 306 for a user.

FIG. 12 illustrates a machine 310 having a memory containing data usedto train a machine learning based affective response model 312 for auser. FIG. 13 illustrates one embodiment of a method for generating themachine learning based affective response model 312. The methodincluding the following steps: In step 320, receiving token instancesrepresenting stimuli that influence the user's affective state. Thetoken instances are spread over a long period of time that spansdifferent situations and have overlapping instantiation periods. In step322, receiving data representing levels of user attention in some of thetoken instances used to improve the accuracy of the machine learningbased affective response model for the user. In step 324, receivingannotations representing emotional states of the user. The annotationsare spread over a long period of time that spans different situations.And in step 326, training the machine learning based affective responsemodel for the user on the token instances, the data representing levelsof user attention in some of the token instances, and the annotations.

Predictors

In one embodiment, a machine learning-based predictor is trained forpredicting the user's response when exposed to token instances.Optionally, the predictor predicts the user's affective response whenexposed to the token instances. Optionally, the predictor predicts thevalues corresponding to one or more of the user's measurement channels.Optionally, the predictor may utilize any known or yet-to-be inventedmachine learning methods for classification or prediction, which operateon data samples and return a predicted target value.

In one embodiment, a machine learning training procedure is suppliedtraining data comprising of samples and corresponding labels or targetvalues. The samples include information derived from token instances.Optionally, samples are derived from temporal token instance windows,for example, by using a vector representation for the windows.Optionally, the samples are preprocessed in various ways, for example,normalizing, filtering, and/or binning some of the values. Optionally,samples are augmented with additional information, for example, baselinevalues, user measurement channel values, values describing the distancefrom a baseline, values describing counts of instances in the temporaltoken instance window (in order to account for saturation), and/orvalues corresponding to previous instantiation of some of the tokens (inorder to account for habituation). Optionally, some samples are assignedweight values, for example, in order for the machine learning proceduresto emphasize them appropriately in the training.

In one embodiment, the data used to create the samples for training amachine learning based predictor is collected by monitoring a user overa long period of time (for instance hours, days, months and even years),and/or when the user was in a large number of different situations.Optionally, the training data is comprised of token instancesoriginating from multiple sources of different types. For example, sometoken instances comprise representations of elements extracted fromdigital media content. In another example, some token instances compriserepresentations of elements extracted from an electromechanical devicein physical contact with the user. Optionally, the training data iscomprised of some token instances with overlapping instantiationperiods, i.e., the user may be simultaneously exposed to a plurality oftoken instances. Optionally, the user may be simultaneously exposed to aplurality of token instances originating from different token sourcesand/or different types of token sources.

In one embodiment, a machine learning-based predictor is trained forpredicting the user's response when exposed to token instances.Optionally the response may be given in the form of a value of acategorical variable. Optionally, the response may be given in the formof a value for scalar variable, such as an integer or real value.Optionally, the response may be given in the form of a value of amultidimensional variable.

In one embodiment, a machine learning-based predictor for a user'smultidimensional response value may be obtained by merging the outcomeof multiple predictors for single dimensional response values,corresponding to individual dimensions of the desired multidimensionalresponse. In some cases, there are correlations between the dimensionsof a multidimensional response, such as when the response is anaffective response or the response is given in the form of usermeasurement channel values. Therefore, in one embodiment, themultidimensional response is predicted in a two stage approach. First, amodel for each response dimension is trained independently. In thesecond stage, a model for each response dimension is trained, whereinthe response values for the other dimensions are also provided as aninput. The final response is obtained by merging the results from thepredictions of the models trained at the second stage. Optionally, amultidimensional predictor may utilize single dimensional predictorsusing the method of output-associative fusion, as described in Nicolaou,M. A., Gunes, H., & Pantic, M. (2011) Continuous Prediction ofSpontaneous Affect from Multiple Cues and Modalities in Valence—ArousalSpace, IEEE Transactions on Affective Computing, which describes amethod in which the correlations between dimensions may be leveraged toincrease the accuracy of a multidimensional prediction.

In one embodiment, some of the samples used for training the machinelearning-based predictor do not have corresponding labels or targetvalues. In this case, training may be performed using semi-supervisedmachine learning techniques. Often semi-supervised methods are ableutilize unlabeled samples, in order to gain additional accuracy.Optionally, different methods for semi-supervised training are used totrain more accurate predictors, such as the methods discussed in Zhu, X.and Goldberg, A. (2009), Introduction to semi-supervised learning.Morgan & Claypool Publishers, which describe various approaches in whichthe unlabeled data may be utilized in the learning process, such as (i)mixture models in which the model's parameters are learned also from theunlabeled data using an expectation maximization (EM) algorithm; (ii)self-training (also referred to as bootstrapping), wherein the predictoror classifier is used to assign target values to unlabeled samples, andis thus able to increase the body of labeled samples from which it canlearn; (iii) co-training, wherein two or more learners are trained on aset of examples and used to classify unlabeled samples, but with eachlearner using a different sets of features.

In one embodiment, in which there are many more training samples thantarget values or labels, the target values may be collected or receivedintermittently. Optionally, by “intermittently”, it is meant that thereare periods of times in which target values or labels are available, andthose periods may be separated by periods of time in which target valuesor labels are not available. Optionally, by “intermittently”, it is alsomeant that the target values or labels may appear sporadically at times,i.e., single target values or labels may be available at certain times,separated by periods in which there are no target values or labelsavailable.

In one embodiment, the machine learning-based predictor for the user'sresponse to tokens is created by using ensemble methods that aggregatethe results of different models. For example, methods applying boostingor bagging.

In one embodiment, various dimensionality reduction feature extractionmethods may be used to reduce the data's dimensionality, such asPrincipal Component Analysis (PCA), or Local Linear Embedding (LLE). Inone embodiment, feature selection methods may be used in order to reducethe data dimensionality and remove dimensions that are not relevant tothe prediction task.

In one embodiment, a Naive Bayes model is trained on labeled trainingsamples. Optionally, the Naive Bayes model is used as a classifier,returning a categorical response value. Optionally, some of thevariables in the samples are converted into binary variables, such thatall non-zero values are set to one. Optionally, the values of thevariables in the input data are binned, such that the variables areconverted to discrete multinomial variables. Optionally, some of thevariables are assumed to be distributed according to a parametricdistribution such as the Normal distribution. Optionally, the trainedNaive Bayes model is comprised of class prior probabilities and classconditional probabilities; the class prior probabilities describe theprior probability for a sample to be labeled with a specific category;the class conditional probabilities describe the probability for avariable to have a specific value given the sample is labeled with aspecific label (class).

In one embodiment, a Naive Bayes model is trained using both labeled andunlabeled data. Optionally, the Naive Bayes model is used as aclassifier that predicts a categorical response value. Optionally, themodel is trained using an Expectation Maximization algorithm comprisingthe following steps:

(i) Training a Naive Bayes classifier using only the labeled samples toobtain a set of parameters that includes the initial class prior andclass conditional probabilities.

(ii) Repeating the following Expectation-Step and Maximization-Stepwhile the classifier's parameters improve the performance, e.g., byreducing the classification error rate on an independent test set: (a)Expectation-Step. Using the current classifier parameters, compute forall samples (both labeled and unlabeled) the probability that thesamples belong to each of the classes (these probabilities are referredto as “component probabilities”). (b) Maximization-Step. Re-estimate theclassifier parameters from all samples using the updated componentprobabilities.

(iii) Outputting the parameters with which classifier obtained the bestperformance.

Optionally, the Naive Bayes model trained in a semi-supervised methodcomprises class prior probabilities and class conditional probabilities;the class prior probabilities describe the prior probability for asample to be labeled with a specific category; the class conditionalprobabilities describe the probability for a variable to have a specificvalue given the sample is labeled with a specific label (class).

In one embodiment, a maximum entropy model is trained to be used as aclassifier that predicts a categorical response value. Maximum entropymodels are a multiclass extension of logistic regression models.Optionally, a maximal entropy model uses feature functions of the formf(x,c), where x is an input variable and c is a class. For example, fora certain sample, the value of f(x,c) may behave as follow, if thesample is labeled by class c, f(x,c) returns the value of feature x,otherwise f(x,c) returns 0. A maximal entropy model comprises weightingparameters λ_(i,j), for 1≦i≦N, and 1≦j≦C, that correspond to the N×Cfeature functions used to train the model (assuming the input vectorshave N features and there are C categories to predict). More informationon Maximum entropy models and their training is available, for example,in Berger, A. L. Della Pietre, S. A. Della Pietra, V. J. (1996) MaximumEntropy Approach to Natural Language Processing. ComputationalLinguistics, 22(1), pages 39-72.

In one embodiment, a neural network model is trained in order to serveas a predictor of a categorical response value, a single dimensionalresponse value, or a multidimensional response value. Optionally, theneural network comprises of an input layer or neurons, one or morehidden layers of neurons, and an output layer of neurons. Optionally,the neural network may utilize a feedforward topology. Alternatively,the neural network may be an Elman/Jordan recurrent neural networktrained using back-propagation.

In one embodiment, a random forest is trained in order to serve as aclassifier. A random forest is an ensemble method that aggregates thepredictions of many decision trees. More information on random forestsis available, for example, in Breiman, Leo (2001). “Random Forests”.Machine Learning 45 (1): 5-32.

In one embodiment, a regression model is used as a predictor of a singledimensional response variable. Optionally, the regression technique usedis Ordinary least squares. Optionally, the regression technique used isweighted least squares (to account for weighted input samples).Optionally, the regression technique used is least angle regression(which has been shown to work well with high-dimensional data).Optionally, the regression technique used is LASSO regression (whichincludes regularization terms).

In one embodiment, the regression may take the form y=X·β+ε, where y isthe response vector (for example, emotional state arousal values), X isa matrix whose rows are the vectors representing the samples (forexample, vector representations of temporal token instance windows), βis the model parameter vector, and ε is the error vector. The goal ofthe training is to minimize the squared error of the difference betweeny and X·β.

In one embodiment, regression models are used for predicting amultidimensional response value. Optionally, the multidimensionalprediction is done by training separate regression models for each ofthe predicted dimensions. Optionally, Multiple Response Regression maybe used, as described in Hastie, T., Tibshirani, R. and Friedman, J.(2001) The Elements of Statistical Learning. Springer, which describes aregression technique that can leverage the correlations betweendifferent dimensions of the response values.

In one embodiment, a support vector machine for regression (SVR) is usedas a predictor for a single dimensional response value.

In one embodiment, a Support Vector Machine (SVM) is trained in order tobe a predictor for a categorical response value.

In one embodiment, an Input-Output Hidden Markov Model (IOHMM) istrained in order to be a predictor of a multidimensional response value.Like a Hidden Markov Model (HMM), an IOHMM has hidden discrete statevariables. However, unlike HMM's, IOHMM also receives input values (inaddition to emitting outputs). Each state may use a predictor, such as aneural network to predict the output values given the input values. Inaddition, similar to HMM, the model also maintains transitionprobabilities to move between states. More information on IOHMMs isavailable, for example, in Bengio, Y. and Frasconi, P. (1996).Input/output HMMs for sequence processing. IEEE Trans. on NeuralNetworks, 7(5):1231-1249. Optionally, some of the training samples mayhave state assignments, for example, corresponding to situationidentifiers, which can be used to initialize some of the parameters inthe expectation maximization algorithm used to train the model'sparameters.

In one embodiment, a predictor for a user's response after being exposedto a stream of token instances utilizes a machine learning-basedprediction algorithm. Optionally, the predicted user response is theuser's affective response. Optionally, the predicted user response is avalue of a measurement channel of the user.

In one embodiment, training data used to train a machine learning-basedpredictor for a user's response after being exposed to a stream of tokeninstances is collected by monitoring the user when he/she is exposed toa plurality of streams of token instances, optionally with the streamshave varying durations. Optionally, the training data is collected bymonitoring the user over a long period of time (for instance hours,days, months and even years), and/or when the user was in a large numberof different situations. Optionally, the training data is comprised oftoken instances originating from multiple sources of different types.For example, some token instances comprise representations of elementsextracted from digital media content. In another example, some tokeninstances comprise representations of elements extracted from anelectromechanical device in physical contact with the user. Optionally,the training data is comprised of some token instances with overlappinginstantiation periods, i.e., the user may be simultaneously exposed to aplurality of token instances. Optionally, the user may be simultaneouslyexposed to a plurality of token instances with overlapping instantiationperiods, possibly originating from different token sources and/ordifferent types of token sources.

The following three methods can utilize the aforementioned training datato train a machine learning-based predictor to predict the user'sresponse after being exposed to a stream of token instances.

(i) In one embodiment, a method for predicting a user's response afterbeing exposed to a stream of token instances represents a stream oftoken instances using a single temporal window of token instances. Thestream of token instances is converted into a single vector representinga temporal token instance window with a length approximately matchingthe duration of the token instance stream. Optionally, some of thevalues in the vector representing the temporal token instance window,such as token instance weights, are normalized according to the durationof the stream of token instances. For example, the weights are dividedby the duration of the stream in minutes, in order to be compatible withwindows representing streams of other durations. Optionally, theduration of the token instance stream is provided as an additional inputvalue as part of the vector representation of the temporal tokeninstance window. Optionally, additional information is incorporated intothe vectors of the training data, such as variables identifying thesituation in which the user is at the time, and/or variables describinga baseline level.

In one embodiment, the training data comprising a plurality of tokeninstance streams to which the user was exposed is converted intotemporal token instance windows, of possibly variable durations (onewindow per token instance stream), and the user's response is notedbefore and after the exposure to the stream of tokens. Optionally, theuser's response before the exposure to the stream of tokens may beprovided as part of the sample's data, while the response after theexposure to the token instance stream may serve as the target value forprediction. Optionally, the difference between the response after theexposure and the response before the exposure to the stream of tokeninstances is used as the target value for prediction.

In one embodiment, training data is collected and used to train amachine learning-based predictor such as a recurrent neural network,support vector machine for regression, or regression model. Given a newstream of tokens, and optionally, the user's response before exposure,the trained predictor may be used to predict the user's response afteran exposure to the stream of token instances, by converting the streamof token instances into a single vector representing a temporal tokeninstance window, and returning the predictor's result on the singlevector.

(ii) In one embodiment, a method for predicting a user's response afterbeing exposed to a stream of token instances uses a representation forthe stream of token instances as multiple vectors; wherein the vectorsrepresent consecutive temporal token instance windows of a substantiallyfixed duration, for example 10 seconds. Training data comprising aplurality of token instance streams to which the user was exposed iscollected and converted into temporal token instance windows of fixedduration. The user's response is noted both at the beginning of temporaltoken instance windows, and at their end. Optionally, the user'sresponse at the beginning of the window may be provided as part of thesample vector data, while the response at the end of the window mayserve as the target value for prediction. Optionally, the differencebetween the response at the beginning and end of the window may be usedas the target value for prediction. Optionally, additional informationis incorporated into the vectors of the training data, such as variablesidentifying the situation in which the user is at the time, and/orvariables describing a predicted baseline level.

In one embodiment, training data is collected and used to train amachine learning-based predictor such as a recurrent neural network,support vector machine for regression, or regression model. Given a newstream of tokens, and optionally, the user's response before exposure,the model may be used to predict the user's response after the exposureto the stream of token instances. The stream of token instances isconverted into a sequence of vectors representing consecutive temporaltoken instance windows of a substantially fixed duration. The predictionmay proceed as follows. At the first step, the value of the user'sresponse before exposure to the stream of tokens instances is used forthe user's response value at the beginning of the first window. Theuser's response value after exposure to the first window is thenpredicted using the trained predictor. This predicted response thenserves as the value of the user's response before the exposure to thesecond window and so on. Thus, the predicted response to window i servesas the initial response value of window i+1. This process proceeds untilthe user's response, after being exposed to the token instances of thelast window, is predicted. This last predicted response value isreturned as the user's response to being exposed to the entire stream oftoken instances.

(iii) In one embodiment, a method for predicting a user's response afterbeing exposed to a stream of token instances uses a Hidden Markov Model(HMM). This method uses categorical state values to represent the userresponse, for example, emotional state categories such as anger,happiness, sadness, excitement, surprise. A stream of token instances isrepresented as multiple vectors; wherein the vectors representconsecutive temporal token instance windows of a fixed duration, forexample 10 seconds. Training data comprising one or more token instancestreams to which the user was exposed is collected and converted intotemporal token instance windows of a fixed duration. The user'scategorical state value during each window is also noted and used as thelabel for the windows. Optionally, additional information isincorporated into the samples of the training data, such as variablesidentifying the situation in which the user is at the time, and/orvariables describing a predicted baseline level.

In one embodiment, collected training data is used to train a HiddenMarkov Model (HMM), in which the hidden states represent the categoricalstate values used to label the windows and the emitted symbols are thetoken instances observed in each window. The training data comprisessets of sequences of windows created from streams of token instances.Training the HMM involves finding a maximum likelihood estimate forparameters such as transition probabilities between states and theemission probabilities (the probability of observing token instances inthe different states). Optionally, the parameters are set using aforward-backward algorithm, such as the Baum-Welch algorithm that usesan Expectation Maximization approach. Given a new stream of tokens, theHMM can be used to predict the user's response after being exposure tothe stream as follows. First, the stream is converted into a sequence ofvectors representing temporal token instance windows of a fixedduration. Then a dynamic programming algorithm, such as the Viterbialgorithm, may be used to predict the most likely final state, inaddition to the most likely path of states in which the user may be ifexposed to the stream of token instances. Optionally, the user's initialstate, which is the response level before exposure to the stream oftoken instances, may be used as the first state in the predicted path ofstates. Otherwise, the dynamic programming algorithm may consider allstates as possible initial states according to prior probabilitieslearned during the HMM model training.

Libraries

In one embodiment, a machine learning-based user response model isanalyzed in order to generate a library of the user's expected responseto tokens representing stimuli to which the user may be exposed. In oneembodiment, the user's response is expressed as an expected affectiveresponse and/or expected change to the user's affective response. In oneembodiment, the response is expressed as the expected value and/orchange in value for one or more user measurement channels. Optionally,the training data used to generate the model comprises of samplesgenerated from temporal token instance windows and target valuescorresponding to the token windows, which represent the user's responseto the token instances from the temporal window of token instances.

In one embodiment, a library of the user's expected response to tokenscomprises various values and/or parameters extracted from the user'smachine learning-based user response model. Optionally, the extractedvalues and/or parameters indicate the type and/or extent of the user'sresponse to some tokens. Optionally, the extracted values and/orparameters indicate characteristics of the user's response dynamics, forexample, how the user is affected by phenomena such as saturation and/orhabituation, how fast the user's state returns to baseline levels, howthe response changes when the baseline is at different levels (such aswhen the user is aroused vs. not aroused).

In one embodiment, a library comprises values generated by monitoringthe results of experiments in which a user's machine learning-based userresponse model was run on various data samples. For example, experimentsin which specific samples (for example, with specific token instances)were run through the model in order to gain insights into the user'sresponse to the specific token instances. Optionally, the experimentsmay involve perturbed data, for example, by modifying the weight ofcertain token instance in order to observe how the response changes as afunction of the modified input level.

In one embodiment, a library of the user's expected response to tokensis generated, wherein the response is expressed as a multidimensionalvalue. For example, the response may be an affective responserepresented as a point or vector in a multidimensional space. In anotherexample, the response may be the value of a plurality of usermeasurement channels. Optionally, the library may comprise responses totoken, expressed as multidimensional values, for example a point or avector in a multidimensional space. Optionally, the library may compriseseparate responses for the token for each of dimension in themultidimensional space used to represent the response value.

In one embodiment, the machine learning-based user response model usedto generate the library of the user's expected response to tokens istrained on data collected by monitoring a user over a long period oftime (for instance hours, days, months and even years), and/or when theuser is in a large number of different situations. Optionally, thetraining data is comprised of token instances originating from multiplesources of different types. For example, some token instances compriserepresentations of elements extracted from digital media content. Inanother example, some token instances comprise representations ofelements extracted from an electromechanical device in physical contactwith the user. Optionally, the training data is comprised of some tokeninstances with overlapping instantiation periods, i.e., the user may besimultaneously exposed to a plurality of token instances. Optionally,the user may be simultaneously exposed to a plurality of token instancesoriginating from different token sources and/or different types of tokensources. Optionally, some of the token instances originate fromdifferent token sources, and are detected by the user using essentiallydifferent sensory pathways (routes that conduct information to theconscious cortex of the brain).

In one embodiment, the training data collected by monitoring the user,is collected during periods in which the user is in a large number ofdifferent situations. Optionally, the data is partitioned into multipledatasets according to the different sets of situations in which the userwas in when the data was collected. Optionally, each partitionedtraining dataset is used to train a separate situation-dependent machinelearning-based user response model, from which a situation-dependentlibrary may be derived, which describes the user's expected response totokens when the user is in a specific situation.

In one embodiment, data related to previous instantiations of tokens isadded to some of the samples in the training data. This data is added inorder for the trained machine learning-based user response model toreflect the influences of habituation. Thus, the library generated fromthe machine learning model may be considered a habituation-compensatedlibrary, which accounts for the influence of habituation on the user'sresponse to some of the token instances. In some cases, habituationoccurs when the user is repeatedly exposed to the same, or similar,token instances, and may lead to a reduced response on the part of theuser when exposed to those token instances. By contrast, in some casesthe user's response may gradually strengthen if repeatedly exposed totoken instances that are likely to generate an emotional response (forexample, repeat exposure to images of a disliked politician).

To account for the aforementioned possible influence of the user'sprevious exposures to tokens, in one embodiment, certain variables maybe added explicitly to some of the training samples. Optionally, theadded variables may express for some of the tokens information such asthe number of times the token was previously instantiated in a giventime period (for example, the last minute, hour, day, or month), the sumof the weight of the previous instantiations of the token in the giventime period, and/or the time since the last instantiation of the token.Optionally, the habituation-related information may be implicit, forexample by including in the sample multiple variables corresponding toindividual instantiations of the same token in order to reflect the factthat the user had multiple (previous) exposures to the token.

In one embodiment, a classifier is provided in order to classify some ofthe tokens into classes. For example, two token instances representingimages of people may be classified into the same class. Optionally,information may be added to some of the training samples, regardingprevious instantiations of tokens from certain classes, such as thenumber of times tokens of a certain class were instantiated in a giventime period (for example, the last minute, hour, day, or month), the sumof the weight of the previous instantiations of tokens of a certainclass in the given time period, and/or the time since the lastinstantiation of any token from a certain class.

In one embodiment, data related to the collection of token instances theuser is exposed to simultaneously, or over a very short duration (suchas a few seconds), is added to some of the samples in the training data.This data is added so the trained machine learning-based user responsemodel, from which the library is generated, will be able to model theinfluence of saturation on the user's response, and thus creating asaturation-compensated library. In some cases, saturation occurs whenthe user is exposed to a plurality of token instances, during a veryshort duration, and may lead to a reduced response on the part of theuser (for instance due to sensory overload). Therefore, in oneembodiment certain statistics may be added to some of the trainingsamples, comprising information such as the number token instances theuser was exposed to simultaneously (or during a short duration such astwo seconds) and/or the weight of the token instances the user wasexposed to simultaneously (or in the short duration). Optionally, aclassifier that assigns tokens to classes based on their type can beused in order to provide statistics on the user's simultaneous (or nearsimultaneous) exposure to different types of token instances, such asimages, sounds, tastes, and/or tactile sensations.

In one embodiment, the machine learning-based user response model usedto generate a library of the user's expected response to tokens, istrained on data comprising significantly more samples than targetvalues. For example, many of the samples comprising temporal tokeninstance windows do not have corresponding target values. Thus, most ofthe samples may be considered unannotated or unlabeled. Optionally, theuser's machine learning-based user response model is trained using asemi-supervised machine learning training approach such asself-training, co-training, and/or mixture models trained usingexpectation maximization. In some cases, the models learned bysemi-supervised methods may be more accurate than models learned usingonly labeled data, since the semi-supervised methods often utilizeadditional information from the unlabeled data, thus being able tocompute things like distributions of feature values more accurately.

In one embodiment, a library of the user's expected response to tokensmay be accessed or queried using various methods. In one example, thelibrary may be queried via a web-service interface. Optionally, theweb-service is provided a user identification number and an affectiveresponse, and the service returns the tokens most likely to elicit thedesired response. Optionally, the system is provided a token, and thesystem returns the user's expected response. Optionally, the service isprovided a token, and the system returns a different token expected toelicit a similar response from the user.

In one embodiment, a Naive Bayes model is trained in order to create alibrary of a user's expected affective response to token instancesrepresenting stimuli. Optionally, the affective response is expressedusing C emotional state categories. Optionally, the library comprisesprior probabilities of the form p(c), 1≦c≦C, and class conditionalprobabilities of the form p(k|c), where k is an index of a token from 1to N (total number of tokens). Optionally, the probability p(c|k) iscomputed using Bayes rule and the prior probabilities and the classconditional probabilities. Optionally, for each class, the tokens aresorted according to decreasing probability p(c|k), thus the library maycomprise ranked lists of tokens according to how likely (or unlikely)they are to be correlated with a certain emotional states with the user.

In one embodiment, a maximum entropy model is trained in order to createa library of the use's expected response to token instances representingstimuli. Optionally, the model comprises the parameters λ_(i,j), for1≦i≦N, and 1≦j≦C, that correspond to the N×C feature functions used totrain the model (assuming the input vectors have N features and thereare C emotional state categories to classify to), and creating j listsof the form λ_(1,j) . . . λ_(N,j), one for each emotional state classj=1 . . . C. Optionally, For each class j=1 . . . C the parametersλ_(1,j) . . . λ_(N,j) are sorted according to decreasing values; the topof the list (most positive λ_(i,j) values) represents the mostpositively correlated token instances with the class (i.e., beingexposed to these token instances increases the probability of being inemotional state class j); the bottom of the list (most negative λ_(i,j)values) represents the most negatively correlated token instances withthe class (i.e., being exposed to these token instances decreases theprobability of being in emotional state class j). Optionally, some inputvariables (for example, representing token instances) are normalized,for instance to a mean 0 and variance 1, in order to make the weightsassigned to feature functions more comparable between token instances.

In one embodiment, a regression model is trained in order to create alibrary of the use's expected single dimensional real-valued response totoken instances representing stimuli. Optionally, the model comprisesthe regression parameters β_(i), for 1≦i≦N, that correspond to the Npossible token instances included in the model. Optionally, theparameters β₁, . . . β_(N) are sorted; the top of the list (mostpositive β_(i) values) represents the token instances that most increasethe response variable's value; the bottom of the list (most negativeβ_(i) values) represents the most negatively correlated token instanceswith the class (i.e., being exposed to these token instances decreasesthe probability of being in emotional state class j). Optionally, someinput variables (for example, representing token instances) arenormalized, for instance to a mean 0 and variance 1, in order to makethe parameters corresponding to different variables more comparablebetween token instances. Optionally, the regression model is amultidimensional regression, in which case, the response for eachdimension may be evaluated in the library separately.

In one embodiment, parameters from the regression model may be used togain insights into the dynamics of the user's response. In one example,a certain variable in the samples holds the difference between a currentstate and a predicted baseline state, for instance, the user's arousallevel computed by a prediction model using user measurement channel vs.the user's predicted baseline level of arousal. The magnitude of theregression parameter corresponding to this variable can indicate therate at which the user's arousal level tends to return to baselinelevels. By comparing the value of this parameter in the user's model,with the values of the parameter in other people's models, insight canbe gained into how the user compares to the general population.

In one embodiment, a neural network model is trained in order to createa library of the use's expected response to token instances representingstimuli. Optionally, the response may be represented by a categoricalvalue, a single dimensional value, or a multidimensional value.Optionally, the neural network may be an Elman/Jordan recurrent neuralnetwork trained using back-propagation. Optionally, the model comprisesinformation derived from the analysis of the importance and/orcontribution of some of the variables to the predicted response. Forexample, by utilizing methods such as computing the partial derivativesof the output neurons in the neural network, with respect to the inputneurons. In another example, sensitivity analysis may be employed, inwhich the magnitude of some of the variables in the training data isaltered in order to determine the change in the neural network'sresponse value. Optionally, other analysis methods for assessing theimportance and/or contribution of input variables in a neural networkmay be used.

In one embodiment, a library comprises of sorting token instancesaccording to the degree of their contribution to the response value, forexample, as expressed by partial derivatives of the neural network'soutput values (the response), with respect to the input neurons thatcorrespond with token instances. Optionally, the list of tokens may besorted according to the results of the sensitivity analysis, such as thedegree of change each token induces on the response value. Optionally,some input variables (for example, representing token instances) arenormalized, for instance to a mean 0 and variance 1, in order to makethe parameters corresponding to different variables more comparablebetween token instances. Optionally, the neural network model used togenerate a response, predicts a multidimensional response value, inwhich case, the response for each dimension may be evaluated in thelibrary separately.

In one embodiment, a random forest model is trained in order to create alibrary of the user's expected response to token instances representingstimuli. Optionally, the response may be represented by a categoricalvalue, for example an emotional state, or categories representingtransitions between emotional states. Optionally, the training data maybe used to assess the importance of some of the variables, for exampleby determining how important they are for classifying samples, and howimportant they are for classifying data correctly in a specific class.Optionally, this may be done using data permutation tests or thevariables' GINI index, as described athttp://stat-www.berkeley.edu/users/breiman/RandomForests/cc_home.htm.

In one embodiment, the library may comprise ranked lists or tokensaccording to their importance toward correct response classification,and towards correct classification to specific response categories.Optionally, some input variables (for example, representing tokeninstances) are normalized, for instance to a mean 0 and variance 1, inorder to make the parameters corresponding to different variables morecomparable between token instances.

FIG. 1 illustrates a machine learning based system for generating alibrary of affective response to tokens 204. Sample generator 114receives token instances 110, and optionally other inputs 113 that mayinclude for example the user's baseline 112, situation data, habituationdata, and/or saturation data. The sample generator 114 generate arepresentation of the token instances and the other value inputs 116.Target value generator 104 receives the user measurement channels 102 ofthe user, and optionally other inputs 103. The Target value generator104 generates affective response annotations 106. A machine learningtrainer 120 receives the representation of the token instances and theother value inputs 116 and the affective response annotations 106, andgenerates an emotional state model 122 that is analyzed by a modelanalyzer 202 to generate the library of affective response to tokens204.

FIG. 2 illustrates a machine learning based system for generating alibrary of user measurement reaction to tokens 224. A machine learningtrainer 170 receives the representation of the token instances and theother value inputs 116 and the user measurement channels 102, andgenerates user measurement model 172 that is analyzed by a modelanalyzer 222 to generate the library of user measurement reaction totokens 224.

FIG. 3 illustrates a machine learning based system with input basedynamic analysis for generating a library of affective response totokens 212. Sample generator 114 receives token instances 110, andoptionally other inputs 113 that may include for example the user'sbaseline 112, situation data, habituation data, and/or saturation data.The sample generator 114 generate a representation of the tokeninstances and the other value inputs 116 that is forwarded to both amachine learning trainer 120 and a model and data analyzer 210. Themachine learning trainer 120 receives representation of the tokeninstances and the other value inputs 116 and the affective responseannotations 106, and generates an emotional state model 122 that isanalyzed by the model and data analyzer 210, together with therepresentation of the token instances and the other value inputs 116, togenerate the library of affective response to tokens 212.

FIG. 4 illustrates a machine learning based system with input basedynamic analysis for generating a library of user measurement reactionto tokens 232. Sample generator 114 receives token instances 110, andoptionally other inputs 113 that may include the user's baseline 112.The sample generator 114 generate a representation of the tokeninstances and the other value inputs 116 that is forwarded to both amachine learning trainer 170 and a model and data analyzer 230. Themachine learning trainer 170 receives representation of the tokeninstances and the other value inputs 116 and the user measurementchannels 102, and generates a user measurement model 172 that isanalyzed by the model and data analyzer 230, together with therepresentation of the token instances and the other value inputs 116, togenerate the library of user measurement reaction to tokens 232.

FIG. 5 illustrates a statistics based system for generating a library ofassociation between tokens and annotations 252. A statistical analyzerof samples and annotations 250 receives the representation of the tokeninstances and the other value inputs 116 and the affective responseannotations 106, and generates the library of association between tokensand annotations 252.

FIG. 6 illustrates a statistics based system for generating a library ofassociation between tokens and user measurements 262. A statisticalanalyzer of samples and measurement 260 receives the representation ofthe token instances and the other value inputs 116 and the usermeasurement channels 102, and generates the library of associationbetween tokens and user measurements 262.

In one embodiment, data collected by monitoring a user over a long time,and optionally, over multiple situations, is used to create an affectiveresponse model for a user. The affective response to a token and/or apattern of tokens may be represented as a probability and/or p-value ofobserving the token or pattern of tokens in a window labeled by acertain emotional category. Alternatively, the affective response to atoken and/or a pattern of tokens may be represented as a probabilityand/or p-value of observing the token or pattern of tokens in a windowwith specific pre-window and post-window emotional states.

In one embodiment, the training procedure for the affective responsemodel utilizes a database of windows D, where the windows may be labeledusing C emotional state categories. Optionally, the database D ispartitioned into C separate window databases (D₁, . . . , D_(C))according to the emotional category used to label the windows.Optionally, the database D is partitioned into C² separate windowdatabases (D_(1,1), . . . , D_(1,C), . . . , D_(C,C)) grouping togetherwindows that have the same pre-window and post-window emotional stateslabels.

In one embodiment, one or more algorithms for finding frequent itemsetsmay be run on each of the N partitioned databases to find the mostfrequent patterns of tokens in each partitioned database. Let F_(i)denote the set of all frequent patterns of tokens found for partitioneddatabase D_(i), the set of all frequent token patterns in D is given byF=F₁UF₂ . . . UF_(N). Let Q denote the set of tokens appearing inpatterns in F, i.e., Q={token k|k∈P, {∈F}.

In one embodiment, the number of windows in the databases D_(i), 1≦i≦N,in which pattern P appears at least once, is denoted by O[P|i].Optionally, the values O[P|i] may be computed efficiently by creating aFrequent Pattern growth tree (FP-tree) for each partition database D_(i)using all the tokens that appear in one or more of the token subsets inF. Optionally, the counts O[P|i] are smoothed, for example by adding ½to all counts, in order to avoid zero probabilities and p-values. Thetotal number of windows in D where pattern P appeared at least once isgiven by O[P]=Σ_(i=1 . . . N)O[P|i]. The number of windows in D_(i) isdenoted O[Di], and the number of windows in D is given byO[D]=Σ_(i=1 . . . N)O[D_(i)].

In one embodiment, a null hypothesis is used for creating the model.Optionally, according to the null hypothesis, token instances do notinfluence the user's emotional state, as represented by the labeledstate of the window and/or the pre- and post-window states. Therefore,under this null hypothesis, the probability of a window being inpartition i, denoted p(i), is given by p(i)=O[D_(i)]/O[D].

In one embodiment, under the null hypothesis, the expected number ofwindows in D_(i), in which a pattern P is observed is given byE[P|i]=p(i)·O[P].

In one embodiment, a statistical test may be used to determine if thedifference between E[P|i] and O[P|i] is statistically significant, whichmay indicate that the null hypothesis is wrong and that the emotionalstate is influenced by the observed token sets. Optionally, thesignificance of a pattern P appearing in D_(i) is computed usingPearson's chi-square test. A p-value for observing P in some of thewindows of D_(i) may be obtained by looking up the Pearson's cumulativetest statistic, which asymptotically approaches a χ² distribution, or byapproximating it using a binomial distribution.

In one embodiment, the probabilities and/or p-values computed forobserving a token or pattern of tokens in windows of various partitionsare used to create an affective response model for the user.

In one embodiment, the affective response model may be used to create alibrary for the affective response to patterns and/or tokens, byincluding one or more of the following:

(i) For each partition i, 1≦i≦N, and pattern P, the probability ofobserving P at least once in a window in the partition D_(i), is givenby p(P|i)=O[P|i]/O[D_(i)]).

(ii) For each partition i, 1≦i≦N, and pattern P, the p-value computedfor observing P. For example, from computing Pearson's test statisticfor the difference between E[P|i] and O[P|i].

(iii) For each partition i, 1≦i≦N, and token k, the probability ofobserving k in a window D_(i), denoted p(k|i)=O[k|i]/O[k|i]/O[D_(i)].Optionally, p(k|i) may denote the probability of observing a tokeninstances with a certain weight in D_(i), in which case it may be givenas p(k=w|i)=w(k,i)/W(i), where w(k,i) is the sum of the weights given toall instances of token k in windows in D_(i) (extracted from the vectorrepresentation of the windows), and W(i) is the weight of all tokens inD_(i), obtained by summing all the weights in the vectors of windows inD_(i). Optionally, values w(k,i) are smoothed by adding a smallpre-defined weight in order to avoid zero-valued probabilities.

Optionally, the lists of patterns and/or tokens with their correspondingprobabilities may be filtered to keep a fixed number of patterns withthe highest probability, and/or keep patterns with a p-value below acertain value.

In one embodiment, some sets of token instances may be replaced bypatterns of more than one token. For example, by scanning the vectorrepresentation of the window and looking for the various patterns in thelibrary. Optionally, the search for patterns may proceed from longestpatterns to the shortest ones, or ordered according to their p-values.Once a pattern of tokens in detected, its instances may be replaced byan instance of a new “pattern token”. Optionally, the attributes of thepattern token may be computed from the attribute values of the instancesthat made it up. For example, the pattern instances weight may equal thesum of its token instance weights, and size may equal the average valueof the size attribute of the token instances. Optionally, a window maybe considered for the purpose of analysis, to contain only token patterninstances, where all token instances not part of larger patterns, areconsidered patterns of one token instance (singletons).

In one embodiment, where the window database D is partitioned into Cpartitions describing the emotional state label given to windows in D,the affective response library may be used to compute the probability ofa window being in state i given patterns of tokens P₁, . . . , P_(n),using Bayes rule as follows:P(i|P ₁ , . . . ,P _(n))=π_(j=1 . . . n) p(P _(j)|i)·p(i)/Σ_(i=1 . . . C)[π_(j=1 . . . n) p(P _(j) |i)·p(i)].

In one embodiment, where the window database D is partitioned into C²partitions describing the pre-window and post-window emotional statelabels given to windows in D, the affective response library may be usedto compute the probability of a user transitioning from emotional statei to state j, 1≦i,j≦C, given the patterns in that window were P₁, . . ., P_(n), may be computed using Bayes rule, as follows:P(i,j|P ₁ , . . . ,P _(n))=π_(j=1 . . . n) p(P _(j)|i,j)·p(i,j)/Σ_(i=1 . . . C)[π_(j=1 . . . n) p(P _(j) |i,j)·p(i,j)]

Therefore, P(j|i, P₁, . . . , P_(n))=P(i,j|P₁, . . . , P_(n))/p(i).

In this description, numerous specific details are set forth. However,the embodiments of the invention may be practiced without some of thesespecific details. In other instances, well-known hardware, software,materials, structures, and techniques have not been shown in detail inorder not to obscure the understanding of this description. In thisdescription, references to “one embodiment” mean that the feature beingreferred to may be included in at least one embodiment of the invention.Moreover, separate references to “one embodiment” or “some embodiments”in this description do not necessarily refer to the same embodiment.Illustrated embodiments are not mutually exclusive, unless so stated andexcept as will be readily apparent to those of ordinary skill in theart. Thus, the invention may include any variety of combinations and/orintegrations of the features of the embodiments described herein.Although some embodiments may depict serial operations, the embodimentsmay perform certain operations in parallel and/or in different ordersfrom those depicted. Moreover, the use of repeated reference numeralsand/or letters in the text and/or drawings is for the purpose ofsimplicity and clarity and does not in itself dictate a relationshipbetween the various embodiments and/or configurations discussed. Theembodiments are not limited in their applications to the details of theorder or sequence of steps of operation of methods, or to details ofimplementation of devices, set in the description, drawings, orexamples. Moreover, individual blocks illustrated in the figures may befunctional in nature and do not necessarily correspond to discretehardware elements. While the methods disclosed herein have beendescribed and shown with reference to particular steps performed in aparticular order, it is understood that these steps may be combined,sub-divided, or reordered to form an equivalent method without departingfrom the teachings of the embodiments. Accordingly, unless specificallyindicated herein, the order and grouping of the steps is not alimitation of the embodiments. Furthermore, methods and mechanisms ofthe embodiments will sometimes be described in singular form forclarity. However, some embodiments may include multiple iterations of amethod or multiple instantiations of a mechanism unless noted otherwise.For example, when an interface is disclosed in an embodiment, the scopeof the embodiment is intended to also cover the use of multipleinterfaces. Certain features of the embodiments, which may have been,for clarity, described in the context of separate embodiments, may alsobe provided in various combinations in a single embodiment. Conversely,various features of the embodiments, which may have been, for brevity,described in the context of a single embodiment, may also be providedseparately or in any suitable sub-combination. Embodiments described inconjunction with specific examples are presented by way of example, andnot limitation. Moreover, it is evident that many alternatives,modifications, and variations will be apparent to those skilled in theart. It is to be understood that other embodiments may be utilized andstructural changes may be made without departing from the scope of theembodiments. Accordingly, it is intended to embrace all suchalternatives, modifications, and variations that fall within the spiritand scope of the appended claims and their equivalents.

What is claimed is:
 1. A method comprising: receiving samples comprisingtemporal windows of token instances to which a user was exposed; thetoken instances spread over at least a day, and a subset of the tokeninstances originating from a same source and having overlappinginstantiation periods; receiving target values representing affectiveresponse annotations of the user in response to the temporal windows oftoken instances; and training an emotional response predictor on inputdata comprising: the samples, the corresponding target values, andvalues indicative of the number of token instances in the temporalwindows of token instances; wherein the emotional response predictorutilizes the values indicative of the number of the token instances tocompensate for non-linear effects resulting from saturation of the userdue to the user being exposed to the token instances in the temporalwindows of token instances including the token instances originatingfrom the same source and having overlapping instantiation periods. 2.The method of claim 1, further comprising using the emotional responsepredictor for predicting emotional state of the user after being exposedto a sample comprising a new temporal window of token instances.
 3. Themethod of claim 1, wherein the token instances are spread over a weekduring which the user is in different situations, and the emotionalresponse predictor captures situation dependent variations in theaffective response of the user.
 4. The method of claim 3, whereintraining the emotional response predictor involves training at least twodifferent machine learning-based emotional response predictors on datacollected over periods where the user was in different situations. 5.The method of claim 1, wherein the training sets parameters of aregression model utilized by the emotional response predictor to makeits predictions.
 6. The method of claim 1, wherein the training setsweights of a neural network utilized by the emotional response predictorto make its predictions.
 7. The method of claim 1, wherein the trainingsets parameters of a support vector machine for regression utilized bythe emotional response predictor to make its predictions.
 8. The methodof claim 1, wherein the training sets parameters of a maximum entropymodel classifier utilized by the emotional response predictor to makeits predictions.
 9. The method of claim 1, wherein the training setsparameters of a random forest model utilized by the emotional responsepredictor to make its predictions.
 10. The method of claim 1, whereinthe training sets parameters of a Naive Bayes classifier utilized by theemotional response predictor to make its predictions.
 11. The method ofclaim 1, wherein the training comprises training multiple predictors;and further comprising combining the predictions of the multiplepredictors using an ensemble method.
 12. The method of claim 1, furthercomprising receiving the token instances and a baseline value for theuser; and generating the samples from the token instances and thebaseline value for the user.
 13. The method of claim 1, furthercomprising receiving the token instances and initial user measurementchannel values representing state of the user at beginning of some ofthe temporal windows of token instances; and generating the samples fromthe token instances and the initial user measurement channel values. 14.The method of claim 1, further comprising receiving the token instancesand initial affective response values representing state of the user atbeginning of some of the temporal windows of token instances; andgenerating the samples from the token instances and the affectiveresponse value for the user.
 15. The method of claim 1, furthercomprising receiving additional samples comprising temporal windows oftoken instances that do not have corresponding target values, andtraining the emotional response predictor utilizing a semi-supervisedtraining method.
 16. A method comprising: receiving samples comprisingtemporal windows of token instances, to which a user was exposed,comprising representations of elements extracted from digital mediacontent; the token instances spread over at least a day, and a subset ofthe token instances originating from a same digital media source andhaving overlapping instantiation periods; receiving target valuescorresponding to the temporal windows of token instances; the targetvalues representing affective response annotations of the user inresponse to the temporal windows of token instances; training a machinelearning-based predictor to predict values representing affectiveresponse of the user to being exposed to token instances; wherein thetraining is performed on data comprising the samples, the correspondingtarget values, and values indicative of the number of token instances inthe temporal windows of token instances; and wherein the machinelearning-based predictor utilizes the values indicative of the number ofthe token instances to compensate for non-linear effects resulting fromsaturation of the user due to the user being exposed to the tokeninstances in the temporal windows of token instances including the tokeninstances originating from the same digital media source and havingoverlapping instantiation periods.
 17. The method of claim 16, whereinthe target values are affective response annotations expressed asemotional responses, and the machine learning-based predictor is amachine learning-based emotional response predictor.
 18. The method ofclaim 16, wherein the target values are derived from user measurementchannels of the user, and the machine learning-based predictor expressesthe predicted response to tokens in terms of values of the usermeasurement channel of the user.
 19. The method of claim 16, furthercomprising receiving the token instances and a baseline value for theuser; and generating the samples from the token instances and thebaseline value for the user.
 20. A method comprising: receiving samplescomprising temporal windows of token instances to which a user wasexposed; the token instances spread over at least a day, and a subset ofthe token instances originating from same source and having overlappinginstantiation periods; receiving target values representing a responseof the user to the temporal windows of token instances expressed asvalues of a measurement channel of the user; and training a machinelearning-based predictor on input data comprising: the samples, thecorresponding target values, and values indicative of the number oftoken instances in the temporal windows of token instances; wherein thepredictor utilizes the values indicative of the number of the tokeninstances to compensate for non-linear effects resulting from saturationof the user due to the user being exposed to the token instances in thetemporal windows of token instances including the token instancesoriginating from the same source and having overlapping instantiationperiods.